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ABOUT  THE  GUIDE 


Project  Overview 

The  Job  Requirements  and  Physical  Demands  Survey  Methodology  (Methodology)  Guide  was 
developed  to  provide  the  Ergonomic  Working  Group  (EWG)  a  technique  that  can  be  used  to 
determine  if  a  Potential  Ergonomics  Problem  Area  (PEP A)  should  be  classified  as  an  Ergonomic 
Problem  Area  (EPRA).  The  Survey  is  a  written  screening  tool  that  can  be  easily  administered  to 
PEPA-designated  work  area  employees  by  Public  Health  technicians. 

This  document  is  formatted  to  provide  the  Public  Health  technician  an  easy-to-use  document. 
The  Administrator’s  Guide  serves  as  the  Survey  “How  To”  Guide  and  contains  information  on 
preparing  to  administer  the  Survey,  actually  administering  the  Survey,  and  scoring  the  Survey. 
Appendices  A  -  D  contain  the  Survey,  a  script  for  the  Survey  administrator  to  follow  when 
administering  the  survey,  and  the  Survey  scoring  sheets.  These  appendices  can  be  taken  directly 
from  this  document  and  photocopied  as  necessary.  Appendix  E  contains  the  scientific  basis  used 
in  the  development  of  the  Job  Requirements  and  Physical  Demands  Survey  Methodology. 

Development  and  Testing  Process 

The  Survey  Methodology  design  was  the  result  of  an  iterative  development  and  testing  process 
that  benefited  from  the  support  and  cooperation  of  Air  Force  personnel  at  several  Air  Force 
installations: 

•  Peterson  AFB,  CO; 

•  Patrick  AFB,  FL; 

•  Malmstrom  AFB,  MT; 

•  Cape  Cod  AS,  MA;  and 

•  Armstrong  Laboratory,  Brooks  AFB,  TX. 


The  development  process  began  with  a  review  of  the  scientific  literature  and  strategically- 
planned  visits  to  USAF  Headquarters  Space  Command  installations.  The  purpose  of  the  review 
was  to  identify  other  screening  tools  or  features  of  other  screening  tools  that  could  be  used  to 
satisfy  the  criteria  established  by  the  Air  Force.  The  site  visits  were  performed  to  maximize 
applicability  of  the  Survey  to  Air  Force  operations. 

The  Survey  incorporated  the  results  of  the  literature  review  and  site  visits,  criteria  established  by 
the  Air  Force,  and  a  series  of  discussions  with  Air  Force-designated  technical  advisors.  An 
iterative  approach  was  used  in  order  to  incorporate  ideas  from  all  Survey  contributors.  Prior  to 
conducting  the  reproducibility  and  validity  testing,  seven  different  versions  of  the  Survey  had 
been  developed. 


The  testing  and  validation  process  was  conducted  in  three  distinct  phases:  usability  testing, 
reproducibility  testing,  and  validity  testing.  Usability  testing  was  performed  to  ensure  that 
Public  Health  technicians  would  be  able  to  use  the  Survey  as  intended.  Reproducibility  testing 
was  performed  to  determine  how  consistently  the  Survey  yielded  the  same  results.  Validity 
testing  was  conducted  to  measure  how  closely  the  results  (e.g..  Survey  Priority  Rank  for  several 
work  areas)  obtained  from  an  experienced  ergonomist  matched  the  results  obtained  from 
administration  of  the  Survey. 

Project  Results 

The  specific  objectives  which  served  as  the  basis  for  the  Survey  development  are  compared  to 
the  actual  Survey  performance  in  Table  1. 

Table  1.  Survey  Development  Criteria  Compared  to  Actual  Survey  Performance 


Design  Feature 

Air  Force  Criteria/Objective 

Actual  Survey  Performance  • 

Ease  of  administration. 

Design  the  Survey  to  be 
administered  to  a  group  of 
assembled  employees  'within  one 
hour. 

Objective  exceeded. 

•  The  Survey  can  be  administered  to  and 
completed  by  a  group  of  assembled 
employees  in  approximately  45  minutes. 

•  In  addition,  to  minimize  employee  time 
and  disruption  of  shop  activities,  the 
Survey  can  be  administered  in  the  shop. 

Ease  of  analysis. 

Design  the  Survey  Methodology  to 
enable  a  Public  Health  technician  to 
analyze  the  data  for  25  work  area 
employees  within  four  consecutive 
hours. 

Objective  exceeded. 

•  The  scoring  process  (e.g.,  determination 
of  Survey  Priority  Rank  for  the  shop)  can 
be  completed  in  less  than  two  hours. 
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Table  1.  Survey  Development  Criteria  Compared  to  Actual  Survey  Performance  (Contd.) 


Design  Feature 

Air  Force  Criteria/Objective 

Actual  Survey  Performance 

Identification  of  potential 
source(s)  of 
musculoskeletal 
discomfort. 

The  Survey  should  provide  a  means 
for  employees  to  identify  specific 
work  processes,  activities,  and  tasks 
which  they  believe  are  related  to 
their  reported  musculoskeletal 
discomfort  and/or  WMD. 

Objective  met  or  exceeded. 

•  Parts  II  (Work  Content)  and  III  (Process 
Improvement  Opportunities)  enable 
employees  to  comment  directly  on  the 
tasks,  tools,  equipment,  materials,  etc., 
that  they  believe  most  relate  to  their 
discomfort,  fatigue,  or  exposure  to 
ergonomic  risk  factors. 

•  In  addition,  employees  can  provide  ideas 
on  improvements  that  they  believe  may 
result  in  a  decrease  in  discomfort,  fatigue, 
and/or  future  WMDs.  This  information 
saves  time  for  Public  Health  and  the  EWG 
since  it  may  help  identify  initial  targets 
for  effective  intervention. 

Determination  of 
Ergonomic  Problem  Area 
(EPRA)  status. 

Results  of  the  Survey  should  help 
the  base  EWG  members  determine 
if  a  PEPA  should  be  classified  as  an 
EPRA. 

Objective  met. 

•  Completion  of  the  Survey  process  for  a 
shop  provides  a  Survey  Priority  Rank 
which  enables  Public  Health  to  make  an 
initial  recommendation  for  EPRA  status. 

•  The  Survey  Priority  Rank,  in 
combination  with  other  considerations 
such  as  past  reported  WMDs, 
organizational  factors,  etc.,  enables  the 
EWG  to  make  the  final  determination  of 
EPRA  status  based  on  interpretation  of 
the  Survey’s  most  common  indicators. 

Prioritization  of  EPRA 
classified  work  areas. 

Results  of  the  Survey  should 
prioritize  EPRA-classified  work 
areas  for  “task-specific”  analyses 
and/or  problem-solving  efforts. 

Objective  met  or  exceeded. 

•  The  numerical  Survey  Priority  Rank  can 
be  used  to  prioritize  EPRA-classified 
work  areas  for  “task  specific”  analyses 
and/or  problem-solving  work. 

•  In  addition,  information  from  the  Work 
Content  and  Process  Improvement 
Opportunities  sections  suggests  “task- 
specific”  targets  for  further  analysis  or 
initial  problem-solving  efforts. 
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Table  1.  Survey  Development  Criteria  Compared  to  Actual  Survey  Performance  (Contd.) 


Design  Feature 

Air  Force  Criteria/Objective 

Actual  Survey  Performance 

Interpretation  of 
ergonomic,  psychosocial, 
and  individual  factors. 

Results  of  the  Survey  should 
provide  an  indication  of  and  the 
relative  importance  of  ergonomic, 
psychosocial,  and  individual  factors 
which  may  be  present  in  the  work 
area. 

Objective  met. 

•  Ergonomic  factors  (e.g.,  risk  factors, 
discomfort  reports)  are  of  primary 
importance  in  determining  the  Survey 
Priority  Rank  for  a  shop. 

•  Results  from  the  Organizational  (i.e., 
psychosocial)  Factors  section  can  be 
used  by  the  EWG  to  determine,  for 
example,  high  levels  of  “job  stress”  that 
may  be  causing  an  increase  in  the 
experience  of  discomfort  and  fatigue. 

•  Results  from  the  Contributing  Factors 
section  can  also  be  used  to  interpret  the 
risk  factors/discomfort  factors-based 
Survey  Priority  Rank.  For  example,  if 
the  Contributing  Factors  score  is  above 
20%,  the  discomfort  rating  may  have 
been  impacted  by  a  high  percentage  of 
employees  with  conditions  that  may 
increase  the  prevalence  of  WMDs. 

Calculation  of  employee- 
reported  discomfort 
prevalence  rates. 

Data  from  the  Survey  should  enable 
Public  Health  to  calculate 
employee-reported  discomfort 
prevalence  rates. 

Objective  exceeded. 

•  Information  from  the  Discomfort  Factors 
section  enables  Public  Health  to 
calculate,  by  body  zone  (e.g., 
shoulder/neck,  hands/wrists/arms, 
back/torso,  legs/feet,  and  head/eyes),  the 
percentage  of  employees  within  a  shop 
who  are  experiencing  or  who  have 
experienced  discomfort  in  the  previous 

12  months. 

•  While  not  a  primary  objective  of  the 
Survey,  this  data  may  be  used  by  Public 
Health  to  determine  whether  or  not  it  is 
likely  that  employees  are  under¬ 
reporting  their  musculoskeletal 
discomfort  or  symptoms  of  WMD. 
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The  Survey  Methodology  provides  the  Air  Force  with  a  tool  that  is  unique  to  the  field  of 
ergonomics.  It  is  the  first  survey  tool  for  which  reproducibility  has  been  reported  to  allow  for  the 
following:  (1)  enables  a  massive  organization  to  systematically  and  quickly,  with  a  minimum  of 
resources,  assess  employee  exposure  to  ergonomic  factors  in  all  types  of  work  environments;  (2) 
results  (Survey  Priority  Rank)  can  be  used  to  establish  overall  priorities  for  further  investigation 
on  the  shop  level,  (3)  functions  and  results  closely  reflect  that  which  would  be  provided  by  an 
experienced  ergonomist;  (4)  results  (Work  Content  and  Process  Improvement  Opportunities)  can 
be  used  to  establish  a  plan  for  specific  follow-up  within  the  higher  priority  shops;  and  (5)  can  be 
used  to  measure  the  potential  impact  of  problem-solving  efforts  that  have  been  completed  within 
a  shop  and  for  all  shops  throughout  a  larger  organization. 
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ADMINISTRATOR’S  GUIDE 


The  Administrator’s  Guide  serves  as  the  “How  To”  Guide  for  using  the  Job  Requirements  and 
Physical  Demands  Survey.  The  Guide  is  divided  into  three  sections  which  provide  an  overview 
of  the  Survey  purpose,  instructions  for  administering  the  Survey,  and  instructions  for  scoring  the 
Survey. 

1.0  OVERVIEW  FOR  THE  PUBLIC  HEALTH  OFFICER  AND  TECHNICIAN 

This  section  provides  an  overview  of  the  Survey  and  describes  its  intended  usage.  It  also 
provides  logistical  information  necessary  for  successful  administration  of  the  Survey. 

1.1  Purpose  of  the  Job  Requirements  and  Physical  Demands  Survey 

The  Job  Requirements  and  Physical  Demands  Survey  (Appendix  A)  has  been  designed  to  assist 
the  Ergonomics  Working  Group  (EWG)  at  your  base  in  determining  if  a  shop,  which  has  been 
previously  classified  as  a  Potential  Ergonomic  Problem  Area  (PEP A),  should  be  classified  as  an 
Ergonomic  Problem  Area  (EPRA).  An  EPRA  is  defined  as  “a  work  area  where  an  association 
can  be  shown  between  ergonomic  risk  factors,  employee-reported  musculoskeletal  discomfort, 
and  employee-reported  work-related  musculoskeletal  disorders  (WMDs).” 

The  Srarvey  allows  you  to  make  decisions  at  the  shop  level. 

Base-wide  completion  of  the  Survey  by  PEPA  employees  will  also  enable  you  to  specifically: 

•  prioritize  EPRA-designated  work  areas  for  more  detailed  task-specific  analysis  and/or 
problem-solving; 

•  calculate  the  prevalence  rate  of  employee-reported  discomfort; 

•  identify  the  relative  importance  of  ergonomic,  psychosocial,  and  individual  factors 
which  may  be  present  in  a  work  area; 

•  identify  work  processes,  activities,  and/or  tasks  that  may  be  related  to  employee- 
reported  musculoskeletal  discomfort  and/or  WMDs;  and 

•  identify  the  possible  influence  of  ergonomic  risk  factors  as  you  complete  AF  Form 
190  investigations. 

The  primary  purpose  of  the  Survey  is  not  to  judge  the  effectiveness  of  the  Air  Force  injury  and 
illness  reporting  system.  However,  completing  the  Survey  process  for  a  particular  work  area 
provides  information  that  should  enable  the  EWG  to  determine  whether  it  is  likely  that 
employees  are  under-reporting  their  musculoskeletal  discomfort  or  symptoms  of  WMDs. 
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1.2  Survey  Design  and  Method  for  Completion 

The  Survey  is  designed  for  you  to  administer  to  an  assembled  group  of  employees.  One 
approach  is  to  administer  the  survey  during  a  scheduled  safety  meeting.  The  Survey 
Administration  Script  (Appendix  B)  includes  a  five-minute  overview  for  you  to  provide  to  the 
participants  prior  to  administering  the  Survey.  You  should  allow  a  minimum  of  45  minutes  for 
the  overview  and  Survey  administration. 

Employees  need  to  respond  to  Survey  questions  based  only  on  their  own  personal  experience. 
There  is  not  necessarily  a  “right”  or  “wrong”  answer  to  any  of  the  Survey  questions. 

1.3  Preparation  and  Logistics 

The  following  sections  contain  answers  to  questions  you  may  have  regarding  planning  the 
Survey  administration. 

1.3.1  Who  is  required  to  complete  the  survey? 

Ideally,  every  employee  in  a  PEPA  shop  should  complete  the  survey.  However,  since  the  survey 
is  optional  and  there  wall  likely  be  some  people  absent,  a  response  rate  of  at  least  80  percent  (%) 
is  suggested  as  a  representative  sample  of  the  shop.  If  an  80%  response  rate  cannot  be  obtained, 
then  the  survey  results  should  be  interpreted  with  caution  due  to  possible  selection  bias. 

1.3.2  Can  I  administer  the  survey  to  more  than  one  shop  at  a  time? 

Everyone  at  a  session  does  not  have  to  be  from  the  same  shop,  although  it  is  often  easier  to 
administer  the  survey  to  one  or  two  shops  at  a  time.  If  everyone  in  a  survey  session  is  from  one 
shop,  you  can  write  some  of  the  shop  identification  information  on  the  survey  forms  before 
copying.  This  makes  the  administration  time  of  the  survey  a  little  shorter  and  insures  that  you 
have  the  right  information  on  each  of  the  forms. 

1.3.3  How  many  people  can  complete  the  survey  at  one  time? 

The  number  of  people  that  can  take  the  survey  at  the  same  time  is  only  limited  by  your 
scheduling  constraints  and  the  size  of  the  meeting  room. 

1.3.4  What  should  I  have  in  the  meeting  room? 

The  meeting  room  should  have  sufficient  seating,  desk  space,  and  pencils. 

1.3. 5  What  should  I  tell  people  before  they  come  to  the  session  ? 

When  scheduling  people  to  participate  in  this  survey,  it  is  recommended  that  you  refer  to  the 
survey  as  an  “occupational  health  survey.”  Placing  an  emphasis  on  “ergonomics”  in  reference 
to  the  survey  may  unintentionally  influence  people’s  responses  as  they  try  to  give  you  what  they 
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think  you  are  looking  for.  On  the  other  hand,  it  is  not  a  secret  that  this  will  be  used  by  the  EWG. 
A  sample  aimouncement  would  be  “All  members  of  the  (PEP A  shop  name)  shop  are  requested  to 
participate  in  a  survey  of  job  requirements  and  physical  demands.  Public  Health  will  be 
conducting  this  survey  on  (date  and  time  and  place).  This  survey  will  take  between  30  and  45 
minutes  to  complete.  Thank  you  in  advance  for  your  participation.” 

2.0  PROCESS  FOR  ADMINISTERING  THE  SURVEY 

This  section  provides  information  about  the  Survey  Administration  Script  (Appendix  B).  It  also 
contains  example  questions  and  suggested  answers  to  those  questions  you  may  be  asked  by 
Survey  participants  during  administration  of  the  Survey. 

2.1  Purpose  of  Overview 

The  purpose  of  your  very  brief  overview  is  to  inform  the  employees  that  they  will  be  completing 
an  occupational  health  survey.  Your  objectives  are  to  tell  them  that  the  survey  is  used  to  obtain 
information  about  their  job  requirements  and  physical  demands  and  to  identify  opportunities  for 
improving  work  in  shops  throughout  the  base.  The  overview  should  be  a  maximum  of  five 
minutes  in  length. 

2.2  Script 

A  Survey  Administration  script  has  been  provided  in  Appendix  B.  Please  follow  the  script 
exactly  to  ensure  a  consistent  message  is  provided  each  time  the  survey  is  administered. 

SPECIAL  NOTE:  Just  as  it  is  important  to  deliver  a  consistent  message  in  the  overview,  it  is 
also  important  to  answer  questions  in  a  uniform  manner.  Casual  comments  or  well-intentioned 
responses  to  employee  questions  may  actually  bias  the  results  and  later  impact  the  conclusions 
that  you  make  during  analysis.  The  following  are  commonly  asked  questions  and  the  appropriate 
responses. 

2.3  Answers  to  Commonly  Asked  Questions 

The  following  sections  contain  the  answers  to  questions  that  Survey  participants  may  ask. 

2.3.1  Survey  Part  I 

This  section  contains  questions  that  may  be  asked  pertaining  to  Part  I  of  the  Survey. 

2.3.1. 1  Is  it  2  to  4  hours  each  day  or  an  average  of  2  to  4  hours  per  day? 

The  answers  are  based  on  approximate  time  frames.  If,  during  the  course  of  a  week,  some  days 
you  spend  more  than  2  hours  and  other  days  you  spend  less  than  2  hours,  base  your  response  on 
what  you  think  is  the  average  amount  of  time. 
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2.3. 1.2  1  only  reach  my  arms  a  little  forward,  does  that  count? 


Yes,  if  you  are  reaching  forward,  with  your  arms  away  from  your  body,  mark  the  amount  of  time 
spent  each  day  in  that  position. 

2.3. 1.3  What  does  “cradle”  a  phone  mean? 

Holding  the  phone  in  place  by  squeezing  the  phone  between  your  shoulder  and  ear. 

2.3.2  Survey  Part  II 

No  questions  are  expected  on  Survey  Part  II. 

2.3.3  Survey  Part  HI 

This  section  contains  a  question  that  may  be  asked  pertaining  to  Part  III  of  the  Survey. 

2.3.3.1  What  if  I  do  some  things  that  are  not  on  this  list? 

Please  write  in  a  brief  description  of  the  activity  and  place  a  mark  in  the  appropriate  work 
frequency  box. 

2.3.4  Survey  Part  IV 

This  section  contains  a  question  pertaining  to  Part  IV  of  the  Survey. 

2.3.4. 1  How  specific  should  1  be  in  describing  the  tasks? 

Please  be  as  specific  as  possible.  This  information  will  be  used  by  Public  Health  and 
Bioenvironmental  Engineering  Services  to  identify  tasks,  tools,  equipment,  etc.,  for  further 
investigation.  For  example,  one  type  of  description  could  be  “removing  and  replacing  files  on 
lower  level  shelves  in  the  medical  records  area.”  Another  could  be  “use  of  the  crimping  tool 
when  splicing  cable.”  Still  another  could  be  “replacing  fuel  line  on  (specific)  aircraft.”  Any 
information  that  you  can  provide  that  will  help  clarify  the  tasks  you  have  in  mind  will  be  useful. 

3.0  INSTRUCTIONS  FOR  COMPLETING  THE  SURVEY  ANALYSIS  PROCESS 

This  section  contains  instructions  for  scoring  the  Surveys.  It  also  contains  helpful  tips  which  will 
make  the  scoring  process  easier. 

3.1  Overview  for  the  Administrator 

The  purpose  of  this  section  is  to  guide  you  through  the  process  for  scoring  the  surveys  and  to 
determine  an  assessment  priority  list  for  potential  EPRA  shops.  The  scoring  process  involves 
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tabulating  individual  responses  from  the  completed  surveys,  using  decision  matrices  to  determine 
the  priority  ratings  and  potential  EPRA  status,  and  using  examples  and/or  descriptions  in  the 
Guide  to  help  you  interpret  the  overall  results.  Your  results  will  be  used  by  the  EWG  to 
determine  EPRA  status  and/or  priorities  for  ergonomic  intervention  throughout  the  base. 

3.2  Planning  and  Logistics 

You  will  need  approximately  two  hours  to  complete  the  scoring  process  for  each  shop.  The 
process  may  be  faster  or  slower  depending  on  the  number  of  employees/Surveys  that  you  must 
process  for  a  particular  shop. 

Before  you  begin  scoring,  you  will  need: 

•  all  of  the  completed  surveys  for  each  shop; 

•  a  pen  or  pencil; 

•  a  calculator; 

•  desk  space  to  spread  out  the  surveys;  and 

•  one  copy  of  the  Scoring  Sheets  and  Scoring  Summary  for  each  shop  to  be  scored. 

3.3  Overview  of  Scoring  Procedures  -  Preparation 

Tally  scores  for  only  one  shop  at  a  time.  You  will  need  to  have  all  of  the  completed  surveys  for 
the  shop  before  proceeding  with  scoring.  Compile  all  of  the  tallying  before  you  begin  calculating 
any  scores.  Points  1  through  7  below  provide  an  overview  of  the  scoring  process.  The  actual 
step-by-step  instructions  that  you  will  use  during  scoring  are  provided  in  section  3.4. 

Point  1:  Count  the  total  number  of  surveys.  You  will  need  this  number  for  calculating 
scores. 

Point  2:  Work  on  one  survey  at  a  time.  Complete  the  tallying  on  all  five  scoring  sheets 
(Pages  1  to  5)  for  that  survey  before  continuing  to  tally  with  the  next  survey. 
Instructions  for  tallying  each  section  are  provided  on  the  respective  scoring  sheets. 

Point  3:  After  all  tallying  is  completed,  follow  the  instructions  on  each  form  to  calculate  the 
ratings,  percentages,  and  scores.  Ratings  will  always  be  verbal  descriptors  “High, 
Medium,  or  Low.”  Percentages  and  scores  will  always  be  numeric. 

Point  4:  Transfer  the  ratings,  percentages,  and  scores  from  the  individual  scoring  sheets 
(Appendix  C)  to  the  Summary  Report  (Appendix  D).  Each  item  is  labeled  with  a 
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section  letter  (A,  B,  C,  etc.)  and  an  item  number  (1,  2,  3,  etc.)  on  both  the  scoring 
sheets  and  the  summary  sheets  to  assist  you  in  transferring  the  scores. 

Point  5:  Follow  the  instructions  on  the  Summary  Report  to  establish  the  Survey  Priority 
Rank.  A  Survey  Priority  Rank  score  of  5  or  higher  indicates  that  a  shop  should  be 
given  EPRA  status.  The  higher  the  Survey  Priority  Rank  score,  the  greater  the 
recommended  priority  for  intervention.  Only  the  EWG,  however,  can  determine 
whether  or  not  a  shop  should  be  upgraded  to  EPRA  status. 

Point  6:  For  each  of  the  Other  Considerations,  examine  the  results,  summarize  the  impacts, 
and  comment  as  necessary  in  the  space  provided.  Focus  your  comments  on  the 
potential  impact  of  the  item  on  the  overall  ergonomic  risk  interpretation  or  strategy 
for  intervention. 

Point  7:  The  EWG  determines  an  intervention  strategy  based  on  the  Survey  Priority  Rank 
and  the  Other  Considerations.  Provide  a  summary  of  the  intervention  strategy  in 
the  space  labeled  “Conclusion  and  Recommendation  Summary.” 

3.4  Specific  Scoring  Procedures 

The  specific  instructions  for  scoring  each  part  are  presented  on  the  scoring  sheets  for  that  part  in 
a  step-by-step  manner.  The  information  provided  in  the  following  section  is  intended  to 
supplement  these  instructions  with  explanations  and  suggestions. 

3.4.1  Part  I.  A.  -  Job  Factors:  Risk  Factor  Ratings  (Appendix  C,  Scoring  Sheets,  Page  1) 

This  section  contains  the  instructions  for  scoring  Part  I.  A.  of  the  Survey. 

3.4.1. 1  Step  1  rOuestions  1-381 

For  the  Risk  Factor  Ratings,  make  only  one  tally  mark  per  survey  in  each  body  area  tally 
box.  Make  your  tally  marks  small  enough  to  allow  room  for  the  entire  shop.  Grouping  your 
tallies  in  sets  of  five  will  make  coimting  easier. 

Score  each  survey  completely  before  proceeding  to  the  next  survey.  For  each  body  area,  count 
the  responses  that  are  either  2-4  or  4-8  hours.  If  that  number  exceeds  the  criteria  number  in  the 
right-hand  comer  of  the  tally  box  for  that  body  area,  place  a  tally  mark.  For  example,  suppose  on 
the  first  survey,  you  coimted  four  responses  for  shoulder/neck  that  were  either  2-4  or  4-8  hours. 
Since  four  is  greater  than  two,  place  a  single  tally  mark  in  the  box.  The  tally  marks  will  indicate 
the  number  of  people  with  concerns  in  the  shop.  CAUTION:  The  criteria  number  you  are 
comparing  against  is  different  for  each  body  area. 
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3.4.1. 2  Step  2  and  Step  3 


These  two  steps  lead  you  through  the  process  of  converting  the  number  of  people  with  concerns 
into  a  percentage  of  people  with  concerns.  This  conversion  to  percentages  allows  you  to  easily 
compare  the  responses  from  shops  with  varying  numbers  of  employees.  In  Step  4  you  will 
convert  these  percentages  into  a  Risk  Factor  Rating. 

3.4.1 .3  Step  4 

Step  4  converts  the  percentage  scores  into  Risk  Factor  Ratings  for  each  body  area.  The  Risk 
Factor  Ratings  are  based  on  how  many  of  the  people  within  a  shop  are  exposed  to  risk  factors  for 
each  body  area.  Compare  the  percentage  to  the  scale  provided  to  determine  the  rating  (Low, 
Medium,  or  High).  Write  the  rating  in  the  box  for  the  appropriate  body  area. 

3.4.2  Part  I.  B.  -  Job  Factors:  Organizational  Factor  Rating  (Appendix  C,  Scoring  Sheets, 
Page  2) 

This  section  contains  the  instructions  for  scoring  Part  I.  B.  of  the  Survey. 

3.4.2.1  Step! 

For  the  Organizational  Factor  Ratings,  make  one  tally  mark  for  each  response  of  agree  or 
strongly  agree.  For  example,  since  there  are  six  questions,  if  an  individual  responds  to  agree  or 
strongly  agree  for  all  six  questions,  make  six  tally  marks.  Keep  your  tally  marks  small  and  in 
groups. 

.3.4.2.2  Step  2 

The  scoring  for  the  Organizational  Factors  contains  one  additional  step  not  present  in  the  Risk 
Factor  scoring.  Divide  the  total  number  of  responses  by  six  in  order  to  compensate  for  marking 
up  to  six  tallies  for  each  survey. 

3.4.2.3  Step  3.  Step  4.  and  Step  5 

These  three  steps  lead  you  through  the  process  of  converting  the  scores  into  a  percentage  of 
people  and  finally  to  a  rating.  Compare  the  percentage  to  the  scale  provided  to  determine  the 
rating  (Low,  Medium,  or  High).  Write  the  rating  in  the  box  for  the  appropriate  body  area. 

3.4.3  Part  I.  C.  -  Job  Factors:  Physical  Effort  (Appendix  C,  Scoring  Sheets,  Page  2) 

This  section  contains  the  instructions  for  scoring  Part  I.  C.  of  the  Survey. 
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3.4.3.1  Stepl 


For  the  Physical  Effort  score,  write  the  score  from  each  survey  into  the  tally  box.  Write  the 
numbers  small,  but  legible,  to  allow  room  for  all  the  responses.  Placing  the  numbers  in  distinct 
rows  or  columns  will  make  the  totaling  process  easier.  A  calculator  is  recommended  for  totaling 
these  numbers. 

3.4.3.2  Step  2  and  Step  3 

The  Physical  Effort  score  is  based  on  the  average  score  for  the  shop.  Steps  2  and  3  lead  you 
through  the  calculation  of  the  average.  The  average  score  is  written  in  the  box. 

3.4.4  Part  II.  D.  -  Discomfort  Ratings  (Appendix  C,  Scoring  Sheets,  Page  3) 

This  section  contains  the  instructions  for  scoring  Part  II.  D.  of  the  Survey. 

3.4.4. 1  Step  1 

For  the  Discomfort  Ratings,  make  only  one  tally  mark  per  survey  in  each  body  area  tally 
box.  Look  at  the  second  and  third  question  for  each  body  area.  Match  the  response  to  “how 
often”  with  the  row  in  the  Criteria  Table.  Match  the  response  to  “how  severe”  with  the  colimm 
in  the  Criteria  Table.  If  the  box  that  matches  the  combination  of  “how  often”  and  “how  severe” 
is  shaded,  place  one  mark  in  the  tally  box  for  that  area. 

For  example:  You  are  tallying  the  shoulder/neck  discomfort  section  for  one  person’s  survey. 
The  person  has  responded  that  on  a  “weekly”  basis  they  have  “moderate”  discomfort.  Since  this 
combination  corresponds  to  a  shaded  area,  place  one  mark  in  the  tally  box. 

3.4.4.2  Step  2.  Step  3.  and  Step  4 

These  three  steps  lead  you  through  the  process  of  converting  the  scores  into  a  percentage  of 
people  and  finally  to  a  rating.  Compare  the  percentage  to  the  scale  provided  to  determine  the 
rating  (Low,  Medium,  or  High).  Write  the  rating  in  the  box  for  the  appropriate  body  area. 

3.4.5  Part  II.  E.  -  General  Question  El.  (Appendix  C,  Scoring  Sheets,  Page  4) 

This  section  contains  the  instructions  for  scoring  Part  II.  E.  -  General  Question  El .  of  the  Survey. 

3.4.5. 1  Step  1 

Place  a  mark  in  the  tally  box  for  each  “yes”  response  to  question  61. 
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3.4.5.2  Step  2 


Write  the  nxunber  of  “yes”  responses  for  the  shop  in  the  E.l  score  box.  This  number  is  not 
converted  to  a  percentage  or  rating  because  it  will  be  compared  with  reported  injuries  and 
illnesses  for  the  shop  in  order  to  identify  potential  under-reporting. 

3.4.6  Part  II.  E.  -  General  Questions  E.2.  to  E.5.  (Appendix  C,  Scoring  Sheets,  Page  4) 

This  section  contains  the  instructions  for  scoring  Part  II.  E.  -  General  Questions  E.2.  to  E.  5.  of 
the  Survey. 

3.4.6.1  Stepl 

For  each  of  the  remaining  general  questions  (question  62-65),  place  a  mark  in  the  corresponding 
tally  box  for  a  “yes”  response  to  any  of  the  questions. 

3.4.6.2  Step  2.  Step  3.  and  Step  4 

These  three  steps  lead  you  through  the  process  of  converting  the  scores  into  a  percentage  of 
people.  Unlike  the  Risk  Factors,  Organizational  Factors,  and  Discomfort  Factors  sections,  do  not 
convert  this  score  to  a  rating. 

3.4. 7  Part  III.  -  Work  Content  (Appendix  C,  Scoring  Sheets,  Page  5) 

This  section  contains  the  instructions  for  transferring  data  from  Part  III.  of  the  Survey  to  the 
Scoring  Sheets.  / 

3.4.7.1  Stepl 

You  Avill  need  to  review  Part  III  of  each  of  the  Surveys.  For  the  first  Survey,  write  down  any 
“Types  of  Work”  that  the  employee  marked  as  “Routine.”  Go  through  the  remainder  of  the 
Surveys  to  identify  other  “Routine”  tasks  that  were  not  identified  on  the  first  Survey. 

3.4.7.2  Step  2 

Go  back  through  the  Surveys  and  make  a  tally  for  each  employee  that  identified  the  same  “Type 
of  Work”  as  “Routine.”  Count  the  tallies  and  write  the  total  in  the  total  box. 

3.4.7.3  Step  3.  Step  4.  and  Step  5 

These  three  steps  lead  you  through  the  process  of  converting  the  totals  into  a  percentage  of 
people.  All  ratings,  percentages,  and  scores  will  be  transferred  from  the  individual  scoring  sheets 
to  the  Summary  Report. 
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3.4.8  Summary  Report  (Appendix  D,  Summary  Report  Sheets,  Pages  1-3) 

This  section  contains  the  instructions  for  preparing  the  Sunraiary  Report. 

3.4.8.1  Step  1  and  Step  2 

Transfer  the  Risk  Factor  and  Discomfort  Ratings  from  pages  1  and  3  of  the  Scoring  Sheets  to  the 
appropriate  boxes  on  the  Summary  Report. 

3.4.8.2  Step  3 

Use  the  Ranking  Matrix  table  to  find  Priority  Scores  for  each  body  zone  and  write  the  scores  in 
the  corresponding  boxes.  Select  the  highest  score  of  all  body  parts  from  Step  3  and  enter  the 
score  into  the  Survey  Priority  Rank  box.  This  score  is  the  Survey  Priority  Rank. 

3.4.8.3  Step  4  and  Step  5 

Write  in  the  Organizational  Factor  Rating  and  Physical  Effect  Factor  Score  from  page  2  of  the 
Scoring  Sheets.  Comment  as  appropriate. 

3.4.8.4  Step  6 

Enter  the  General  Question  percentages  from  page  4  of  the  Scoring  Sheets.  Comment  as 
appropriate. 

3.4.8.5  Step  7 

From  page  5  of  the  Scoring  Sheets,  enter  each  of  the  routine  types  of  work  which  had  shop 
percentage  scores  over  20%. 

3.4.8.6  Step  8 

This  step  requires  a  review  of  Part  IV  of  the  Survey  in  order  to  identify  tasks,  tools,  equipment, 
etc.,  listed  by  employees  as  potential  concerns,  as  well  as  potential  improvement  opportunities. 
Comments  should  be  noted  as  appropriate. 

3.4.8.7  Step  9 

Review  the  injury/illness  history  from  this  shop,  attach  applicable  information,  and  provide 
comments  as  appropriate. 
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3.4.8.8  Step  10 


Enter  the  Shop  Status  in  the  box,  and  write  your  follow-up  recommendations  in  the  space 
provided. 

3.5  Interpretation  of  Results 

3.5.1  Survey  Priority  Rank 

The  Survey  Priority  Rank  provides  you  with  a  numeric  indication  of  the  prevalence  of 
ergonomic  risk  factors  and  WMDs  within  a  shop.  The  Survey  Priority  Rank  is  not  an  indicator 
of  the  severity  of  the  risk.  Additional  analyses  are  required  to  make  that  determination. 

If  the  Survey  Priority  Rank  score  is  5  or  above,  it  indicates  the  presence  of  both  ergonomic 
risk  factors  and  discomfort  for  a  majority  of  the  people  within  a  shop.  Shops  with  a  score  in  this 
range  may  be  designated  as  EPRA  shops  by  the  EWG.  Intervention  priority  is  determined  by  the 
Survey  Priority  Rank  score,  a  higher  score  indicates  a  higher  intervention  priority. 

If  the  Survey  Priority  Rank  score  is  less  than  5,  it  indicates  that  the  majority  of  the  people  in 
the  shop  do  not  have  work-related  discomfort  combined  with  recognized  risk  factors.  A  shop 
could  score  in  this  range  even  if  a  small  portion  of  the  shop  personnel  are  intermittently  exposed 
to  conditions  with  considerable  ergonomic  hazards.  A  review  of  the  job  frequencies  in  Part  III, 
the  comments  in  Part  IV,  and  the  shop  injury  history  may  be  useful  when  searching  for 
hazardous  conditions  with  lower  overall  prevalence. 

3.5.2  Other  Considerations  (Appendix  D,  Summary  Report  Page  2) 

3.5.2.1  Organizational  Factor  Rating  tBI 

A  rating  of  High  in  the  Organizational  Factors  section  indicates  many  of  the  people  in  the  shop 
experience  situations  at  work  that  can  lead  to  a  higher  than  normal  level  of  job  stress.  High 
levels  of  job  stress  can  decrease  job  performance,  increase  the  potential  for  heart  disease,  and 
increase  the  experience  of  pain  and  discomfort.  If  the  Organizational  Factor  Rating  is  High,  it 
suggests  consideration  for  follow-up  job  stress  evaluation  regardless  of  EPRA  status.  If  the 
Organizational  Factor  Rating  is  Medium,  the  EWG  should  consider  job  stress  factors  when 
reviewing  an  EPRA  shop.  If  the  Organizational  Factor  Rating  is  Low,  it  suggests  minimal 
concerns  for  job-related  stress  factors  in  that  shop. 

3.5.2.2  Physical  Effort  Factor  Score  tCi 

The  numeric  score  indicates  the  average  level  of  perceived  exertion.  The  higher  the  score  the 
greater  the  level  of  physiological  exertion  present  within  a  shop.  If  the  Physical  Effort  Factor 
score  is  15  or  higher  this  could  explain  the  presence  of  a  high  discomfort  rating  for  a  shop,  in 
spite  of  a  low  ergonomic  risk  factor  rating.  A  score  in  this  range  suggests  consideration  for 
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follow-up  evaluation  regardless  of  Survey  Priority  Rank.  The  15  corresponds  to  “hard”  on  the 
physical  effort  scale.  This  rating  was  selected  as  a  threshold  since  workers  who  perceive  their 
work  to  be  “hard”  may  be  more  likely  to  report  discomfort  or  to  seek  a  need  for  follow-up.  If  the 
score  is  below  15,  these  numbers  can  be  compared  between  shops  to  assess  the  relative 
physiological  stress  within  each  shop.  Caution:  a  score  in  this  range  does  not  indicate  low 
ergonomic  risk. 

.1.5.2.5  Health  Care  Provider  Visits  tE.ll 

This  score  indicates  the  number  of  people  within  the  shop  who  indicate  that  they  have  sought 
medical  attention  during  the  previous  year  for  work-related  discomfort.  This  number  can  be 
compared  to  injury  and  illness  rates  for  the  shop  to  identify  potential  under-reporting. 

3.5.2.4  Recovery  Time  Score  (E.2) 

This  percentage  provides  a  comparison  with  the  Discomfort  Rating.  If  the  Recovery  Time 
score  is  above  30%,  this  shop  has  likely  been  classified  as  an  EPRA  by  the  Survey  Priority 
Rank.  If  the  shop  was  not  classified  as  an  EPRA,  the  scoring  of  the  Discomfort  Factor  section 
could  be  reviewed  to  verify  accuracy.  Since  this  is  an  alternate  measure  of  discomfort  severity,  a 
shop  with  a  percentage  in  this  range  should  receive  further  investigation  by  the  EWG,  regardless 
of  EPRA  status.  The  30%  threshold  was  selected  as  a  conservative  starting  point  for  further 
evaluation.  If  the  percentage  is  below  30%,  verification  of  Discomfort  Factor  section  scoring  is 
not  necessary.  A  shop  with  a  percentage  in  this  range  may  be  either  an  EPRA  or  a  non-EPRA 
shop  depending  upon  other  factors. 

3.5.2.5  Activity  Interruption  Score  (E.31 

This  percentage  provides  an  additional  comparison  with  the  Discomfort  Ratings.  If  the  Activity 
Interruption  score  is  above  50%,  this  shop  has  likely  been  classified  as  an  EPRA  by  the  Survey 
Priority  Rank.  If  the  shop  was  not  classified  as  an  EPRA,  the  scoring  of  the  Discomfort  section 
could  be  reviewed  to  verify  accuracy.  Since  this  is  an  alternate  measure  of  discomfort  severity,  a 
shop  with  a  percentage  in  this  range  should  receive  further  investigation  by  the  EWG,  regardless 
of  EPRA  status.  The  50%  threshold  was  selected  as  a  conservative  starting  point  for  further 
evaluation.  If  the  percentage  is  below  50%,  verification  of  the  Discomfort  Factor  section 
scoring  is  not  necessary.  A  shop  with  a  percentage  in  this  range  may  be  either  an  EPRA  or  a 
non-EPRA  shop  depending  upon  other  factors. 

3.5.2.6  Previous  Diagnosis  Score  tE.41 

This  percentage  provides  a  mitigating  factor  to  compare  with  the  Discomfort  Rating.  If  the 
Previous  Diagnosis  score  is  above  20%,  the  discomfort  ratings  could  have  been  impacted  by  a 
high  degree  of  people  with  previous  conditions.  If  a  shop  has  more  than  20%  of  the  people  with 
previous  conditions,  the  shop  could  have  a  false  positive  EPRA  determination  based  on  the 
Survey  Priority  Rank.  This  would  be  most  likely  in  cases  where  body  areas  in  the  shop  had  a 


12 


High  rating  for  discomfort  with  either  Medium  or  Low  ratings  for  risk  factors.  If  you  suspect 
that  a  shop  may  have  a  false  positive  EPRA  status,  you  can  re-evaluate  the  shop  by  removing  the 
discomfort  scores  for  people  with  previous  diagnoses  and  re-scoring  the  Discomfort  section  and 
Survey  Priority  Rank.  This  new  score  may  provide  a  better  indication  of  the  current  ergonomic 
hazard  level  within  the  shop.  The  20%  threshold  was  selected  as  a  conservative  starting  point  for 
further  evaluation.  If  the  percentage  is  below  20%,  it  can  be  assumed  that  the  prevalence  of 
previous  diagnosis  within  a  shop  had  a  minimal  impact  on  the  scoring  for  that  shop. 

3.5.2.7  Contributing  Factors  Score  (E.S) 

This  percentage  provides  another  mitigating  factor  to  compare  with  the  Discomfort  Rating.  If 
the  Contributing  Factors  score  is  above  20%,  the  discomfort  ratings  could  have  been  impacted 
by  a  high  degree  of  people  with  conditions  that  increase  the  prevalence  of  cumulative  trauma 
disorders.  A  shop  which  was  ranked  an  EPRA  on  the  basis  of  high  discomfort,  with  either 
medium  or  low  risk  factor  ratings,  may  represent  a  false  positive  ranking.  This  shop  could  be  re¬ 
evaluated  by  removing  the  discomfort  scores  for  people  with  contributing  factors  and  re-scoring 
the  Discomfort  section  and  Survey  Priority  Rank.  This  new  score  may  provide  a  better 
indication  of  the  current  ergonomic  hazard  level  within  the  shop.  The  20%  threshold  was 
selected  as  a  conservative  starting  point  for  further  evaluation.  If  the  percentage  is  below  20%, 
it  can  be  assumed  that  the  prevalence  of  contributing  factors  within  a  shop  had  a  minimal  impact 
on  the  scoring  for  that  shop. 

3.5.2.8  Routine  Types  of  Work 

The  Type(s)  of  Work  that  were  identified  by  at  least  20%  of  shop  employees  is/are  included  here. 
This  information  may  be  used  by  Public  Health  to  identify 
homogeneous  groups. 
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APPENDIX  A 


Job  Requirements  and  Physical  Demands 
Survey 
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JOB  REQUIREMENTS  AND  PHYSICAL  DEMANDS  SURVEY 


Jol»Req(iilreme»t$j%n4Pli3rsical  ; 
Demands  Survey 

Date  (YYMMDD) 

Work|ilnce 

Identifier: 

{use  this  space  for  mechanical  imprint) 

Base 

Organization 

Workplace 

Bldg.  No/Location 

Room/Area 

AFSC/Job  Series 


Gender: 

Female  O 

_ 1 _ 

Male  O 

Work  Group: 

Civilian  O  Grade: 

Military  O  Rank: 

Age  Category: 

20  and  under  O  21-30  O  31-40  O  over  40  O 

Length  of  service  at  this  base:  less  than  one  year  O 

more  than  one  year  O 

Length  of  time  in 

current  shop:  less  than  one  year  O 

more  than  one  year  O 

Have  you  completed  this  questionnaire  before? 

Yes  O  No  O 

16 


Description  of  Work  Activities  and  Shop 


Page  2 


Part  I  -  Job  Factors 


This  section  enables  you  to  describe  what  is  involved  in  your  job.  Indicate  how  long  you  do  this  work  on  approximately 
a  daily  basis. 


A.  DESCRIPTION  OF  WORK 


1- 

Figure  B. 


Figure  C 


SHOULDER/NECK 


1. 


2. 

3. 

4. 

5. 


A 

I  work  with  my  hands  at  or  above  chest  level.  (Figure  A,) . 

.  o 

O 

O 

O 

To  get  to  or  to  do  my  work,  I  must  lay  on  my  back  or  side  and 


work  with  my  arms  up . 

o 

o 

o 

o 

I  must  hold  or  carry  materials  (or  large  stacks  of  files)  during 
the  course  of  my  work . 

o 

o 

o 

o 

I  force  or  yank  components  or  work  objects  in  order  to  complete 
a  task . 

o 

o 

o 

o 

I  reach  or  hold  my  arms  in  front  of  or  behind  my  body  (e.g., 
using  a  keyboard,  filing,  handling  parts,  performing  inspection 
tasks,  pushing  or  pulling  carts,  etc.).  (Figures  B.)  . 

o 

o 

o 

o 

6.  My  neck  is  tipped  forward  or  backward  when  I  work.  {Figure 

C.) . 


O 


o  o 


o 


7.  I  cradle  a  phone  or  other  device  between  my  neck  and  shoulder. 
(Figure  D.) . 


o 


o 


o 


o 


17 


Description  of  Work  Activities  and  Shop 


Page  3 


Part  I  “  Job  Factors  (continued) 


HANDAVRIST/ARM 


8.  My  wrists  are  bent  (up,  down,  to  the  thumb  or  little  finger  side) 

while  I  work.  (Figure  E.)  .  Q 


9.  I  apply  pressure  or  hold  an  item/material/tool  (e.g.,  screw 

driver,  spray  gun,  mouse,  etc.)  in  my  hand  for  longer  than  10 
seconds  at  a  time . 

1 0.  My  work  requires  me  to  use  my  hands  in  a  way  that  is  similar  to 

wringing  out  clothes.  (Figure  F.) . 


o 

o 

o 

o 

o 

o 

o 

o 

1 1 .  I  perform  a  series  of  repetitive  tasks  or  movements  during  the 

normal  course  of  my  work  (e.g.,  using  a  keyboard,  tightening 
fasteners,  cutting  meat,  etc.) . 

12.  The  work  surface  (e.g.,  desk,  bench,  etc.)  or  tool(s)  that  I  use 

presses  into  my  palm(s),  wrist(s),  or  against  the  sides  of  my 
fingers  leaving  red  marks  on  or  beneath  the  skin . 

13.  I  use  my  hand/palm  like  a  hammer  to  do  certain  aspects  of  my 

work . 

14.  My  hands  and  fingers  are  cold  when  I  work . 

15.  I  work  at  a  fast  pace  to  keep  up  with  a  machine  production 

quota  or  performance  incentive . 

16.  The  tool(s)  that  I  use  vibrates  and/or  jerks  my  hand(s)  and 

arms(s) . 

17.  My  work  requires  that  I  repeatedly  throw  or  toss  items . 

1 8.  My  work  requires  me  to  twist  my  forearms,  such  as  turning  a 

screwdriver . 

19.  I  wear  gloves  that  are  bulky,  or  reduce  my  ability  to  grip . 

20.  I  squeeze  or  pinch  work  objects  with  a  force  similar  to  that 

which  is  required  to  open  a  lid  on  a  new  jar . 

21.  I  grip  work  objects  or  tools  as  if  I  am  gripping  tightly  onto  a 

pencil . 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 
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Part  I  -  Job  Factors  (continued) 


Page  4 


Figure  G. 


Figure  L 


BACK/TORSO 


When  I  lift,  move  components,  or  do  other  aspects  of  my  work, 
my  hands  are  lower  than  my  knees.  (Figure  G,) . 

G 

G 

O 

G 

I  lean  forward  continually  when  I  work  (e.g.,  when  sitting,  when 
standing,  when  pushing  carts,  etc.) . 

G 

G 

G 

G 

The  personal  protective  equipment  or  clothing  that  I  wear  limits 

or  restricts  my  movement . 

I  repeatedly  bend  my  back  (e.g,,  forward,  backward,  to  the  side. 

G 

G 

G 

G 

or  twist)  in  the  course  of  my  work . 

G 

G 

G 

G 

When  I  lift,  my  body  is  twisted  and/or  I  lift  quickly.  (Figure  H.) 

G 

G 

G 

G 

27.  I  can  feel  vibration  through  the  surface  that  I  stand  on  or 

through  my  seat . 

28.  I  lift  and/or  carry  items  with  one  hand.  (Figure  L)  . 


O  G  O 
GOO 


G 

G 


29.  I  lift  or  handle  bulky  items .  G  G  G  G 

30.  I  lift  materials  that  weigh  more  than  25  pounds .  G  G  G  G 
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Page  5 


Parti  -  Job  Factors  (continued) 


Figure  J. 


LEGS  /  FEET 


3 1 .  My  work  requires  that  I  kneel  or  squat.  (Figure  J.) . 

.  o 

o 

o 

o 

32.  I  must  constantly  move  or  apply  pressure  with  one  or  both  feet 

(e.g.,  using  foot  pedals,  driving,  etc.) . 

33.  When  I’m  sitting,  I  cannot  rest  both  feet  flat  on  the  floor. 

(Figure  K.)  . 


34.  I  stand  on  hard  surfaces . ; .  O  O  O  O 


o 

o 

o 

o 

o 

o 

o 

o 

HEAD /EYES 


35.  I  can  see  glare  on  my  computer  screen  or  work  surface .  O  O  O  O 

36.  It  is  difficult  to  hear  a  person  on  the  phone  or  to  concentrate 

because  of  other  activity,  voices,  or  noise  in/near  my  work  area.  •  Q  Q  Q  Q 

37.  I  must  look  at  the  monitor  screen  constantly  so  that  I  do  not 

miss  important  information  (radar  scope) .  O  O  O  O 

38.  It  is  difficult  to  see  what  I  am  working  with  (monitor,  paper, 

parts,  etc.) .  O  O  O  O 
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Page  6 

Part  I  -  Job  Factors  (continued) 


B.  ORGANIZATIONAL  FACTORS 


p 

p 

/ 

1 

P 

p 

1 

2 

3 

4 

5 

39. 

I  often  feel  unclear  on  what  the  scope  and 
responsibilities  of  my  job  are . 

o 

o 

o 

o 

o 

40. 

I  often  feel  that  I  have  too  heavy  of  a  workload,  one 
that  I  could  not  possibly  finish  during  an  ordinary 
workday . 

o 

o 

o 

o 

o 

41. 

I  often  feel  that  I  will  not  be  able  to  satisfy  the 
conflicting  demands  of  various  people  around  me . 

o 

o 

o 

o 

o 

42. 

I  often  find  myself  unable  to  get  information  needed 
to  carry  out  my  job . 

o 

o 

o 

o 

o 

43. 

I  often  do  not  know  what  my  supervisor  thinks  of 
me,  how  he/she  evaluates  my  performance . 

o 

o 

o 

o 

o 

44. 

I  often  think  that  the  amount  of  work  I  have  to  do 
interferes  with  how  well  it’s  done . 

o 

o 

o 

o 

o 

C.  PHYSICAL  EFFORT 

45. 

How  would  you  describe  the  physical  effort  required  of  your  job? 

6 

7 

8  9  10  11  12  13 

14 

15 

16  17 

18 

19 

20 

No  exertion 

Extremely  Very  Light  Somewhat 

Hard 

Very 

Extremely 

Maximal 

at  all 

light 

light  hard 

hard 

hard 

exertion 

o 

O 

o  o  o  o  o  o 

o 

o 

o  o 

o 

o 

o 
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Part  II  -  Your  Body’s  Response  to  Work  Demands 


Ct 


C/5 


22 


In  the  past  12  months,  have  you  46.  Yes  O  No  O  49.  Yes  O  No  O  52.  Yes  O  No  O  55.  Yes  O  No  O  58.  Yes  O  No  O 

experienced  any^discomfort,  fatigue,  ^  to  question  49  If  go  to  question  52  If  “no^*,  go  to  question  55  If  **no”,  go  to  question  If  “no”,  go  to  question  61 

numbness,  or  pain  that  relates  to  your 

jobl 


How  often  do  you  experience  47.  Daily  O  50.  Daily  O  53.  Daily  O  56.  Daily  O  59.  Daily 

discomfort,  fatigue,  numbness,  or  pain  Weekly  O  Weekly  O  Weekly  O  Weekly  O  Weekly 

in  this  region  of  the  body?  Monthly  O  Monthly  O  Monthly  O  Monthly  O  Monthly 
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Part  III  -  Work  Content 


The  section  below  will  enable  you  to  describe  the  content  of  the  work  that  you  do  in  your  current  shop. 
Fill  in  the  box  that  describes  how  frequently  you  do  the  task  listed,  based  on  the  following  definitions: 

•  Routine:  Performed  on  three  or  more  days  per  week. 

•  Non-routine:  Performed  two  days  a  week  or  less. 

•  Seasonal:  Performed  only  during  certain  times  of  the  year 

•  Never/NA:  You  do  not  perform  this  type  of  work. 


1  No. 

Tvne  of  Work 

Work  Freauencv 

(Check  one)  1 

Routine 

Non-Routine  Seasonal 

Never/NA 

66. 

abrading 

o 

o 

o 

o 

67. 

baking 

o 

o 

o 

o 

68. 

bolting/screwing 

o 

o 

o 

o 

69. 

calling  (telephone  use) 

o 

o 

o 

o 

70. 

chipping 

o 

o 

o 

o 

71.' " ' 

cleaning  by  hand 

'o'":  ^ 

■:"0 

72. 

cleaning  with  high  pressure  equipment 

-  ^  o 

■^,0' 

o 

73. 

coating/immersiiig 

o 

.  :  ,  o.  .  ^  ■ 

6 

■  -  'O' 

74. 

cooking 

o 

.  .  o  ■  .  ■  ■ 

© 

75. 

-copying  :/• 

:  :0 

o 

o 

76. 

crimping  ■ 

O  ' 

. o 

o 

77. 

cutting/shearing 

o 

o 

o 

o 

78. 

drafting/CAD  system  use 

o 

o 

o 

o 

79. 

drilling 

o 

o 

o 

o 

80. 

driving  (vehicles) 

o 

o 

o 

o 

81. 

excavating 

82. 

filing/geneial  administrative 

o 

■  ■  -O 

83. 

flame  cutting/arc  ctitting 

o. 

o 

'■"'■O'V:-'',. 

84. 

fblding/fitting 

o 

■:  O  "  . 

o 

85.  : 

gluing/lamihating 

o 

g 

86. 

grinding/buffmg/polishing 

o 

o 

o 

o 

87. 

hammering 

o 

o 

o 

o 

88. 

lifting 

o 

o 

o 

o 

89. 

loading  (pallets,  trucks,  carts,  aircraft) 

o 

o 

o 

o 

90. 

lubricating 

o 

o 

o 

o 
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P art  III  “  Work  Content  (Continued) 


1  No. 

Tvne  of  Work 

Work  Freauencv 

1  (Check  one)  1 

Routine 

Non-Routine  Seasonal 

Never/NA 

91. 

machining 

o 

o 

o 

o 

92. 

masoning 

o 

o 

o 

o 

93. 

melting 

o 

o 

o 

o 

94. 

molding 

o 

o 

o 

o 

95. 

monitoring  (visual  displays) 

o 

o 

o 

o 

96. 

mousing  (for  computer  work) 

o 

yy,:';yyoy’iryy 

97. 

nailing 

o 

o 

o 

98. 

opening/closing  heavy  doors 

o 

o 

o 

o 

99. 

packing/packaging 

o 

"O','' 

:■;/  o 

100. 

painting/spray  painting 

vOy"'"',:,'  .-yy"'.,:- 

yp 

.Q 

101. 

paving 

o 

o 

o 

o 

102. 

pumping  (by  hand) 

o 

o 

o 

o 

103. 

riveting/bucking 

o 

o 

o 

o 

104. 

sanding 

o 

o 

o 

o 

105. 

sawing 

o 

o 

o 

o 

106. 

'':;:v"Oyy:yy:;:':-'i;;%y.y,^ 

o 

'  O . 

107. 

sewing 

o 

o 

o 

108. 

soldering/brazing 

o 

o 

o 

o 

109. 

stapling 

^ -O'  ,  . 

o 

o 

110.  i 

stripping/depainting  by  hand 

i'"' ."O 

o 

o 

jlL;^ 

stripping/depainting  mechanically 

,'"  o''" 

^Oyv. 

^.yyyyyO:  :yl;,'^': 

112. 

transporting  loads  on  non-powered  carts 

O 

o 

o 

o 

113. 

turning  valves 

O 

o 

o 

o 

114. 

tying/twisting/wrapping 

o 

o 

o 

o 

115. 

typing/keying 

o 

o 

o 

o 

116. 

welding 

o 

o 

o 

o 

117.^^ 

^::wheeliiig:'lbads '  •  •  : 

■^'~"'‘yy:'0"y"yy'"^"'y' . 

118. 

■'■  wiring 

",  :0' 

O'; ’ yy ^ ^ 

':":--'Oy' 

119. 

wrenching/ratcheting 

^  0-. 

,0 

o;,  ■'^y■y'■yy^: 

'■o.'-'  y  y 

120. 

writing/illustrating 

."y,;VO  ;, 

■y  o',' 

^  ::;,yyyO:,:;,'  ^  :. 

(Write  in  others) 

121. 

o 

o 

o 

o 

122. 

o 

o 

o 

o 
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Part  IV  -  Process  Improvement  Opportunities 

Think  about  your  job  as  a  whole,  including  routine,  non-routine  or  seasonal  work. 

Read  the  questions  listed  below  and  describe  the  activities  that  you  or  your  co-workers  think  place  the  greatest  demands  on  your  body. 
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APPENDIX  B 


Job  Requirements  and  Physical  Demands 
Survey 


Administration  Script 
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(WELCOME  AND  INTRODUCTION) 


Welcome  and  thank  you  for  taking  the  time  to  complete  this  occupational 
health  survey.  The  survey  will  assess  your  job  requirements  and  physical 
demands. 

The  purpose  of  the  Survey  is  to  enable  the  Air  Force  to  better  understand 
and  identify  opportunities  for  improving  work  in  shops  throughout  the  base. 

After  you  complete  the  Survey,  we  will: 

•  analyze  the  results  for  the  entire  shop; 

•  determine  a  Priority  Score  for  the  shop; 

•  provide  information  to  the  Ergonomics  Working  Group. 

We  will  then  decide  on  priorities  for  follow-up  and  shop  improvement. 

This  is  an  anonymous  Survey.  You  will  notice  that  we  do  not  ask  you  to 
provide  your  name  and  there  is  no  coding  system.  The  Survey  is  also 
voluntary;  you  are  not  required  to  take  the  Survey;  however,  your 
participation  is  appreciated. 

We  are  using  the  Survey  to  get  an  overall  assessment  of  the  experiences  in 
your  shop  as  a  whole. 

We  are  not  looking  at  each  person  and  your  individual  responses. 

However,  if  you  wish  to  request  a  follow-up  visit  by  Public  Health,  you  may 
do  so. 
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(OVERVIEW  OF  THE  SURVEY) 

The  Survey  is  divided  into  a  cover  page  and  four  parts. 

I  will  give  you  a  quick  overview  of  each  section  so  follow  along  with  me  as 
I  go  through  the  form. 

The  Cover  Page  asks  for  general  information  about  yourself.  Please  fill  out 
all  of  the  information  on  this  page  with  the  exception  of  the  “workplace 
identifier”  section. 

Turn  to  Page  2. 

Part  I  is  called  “Job  Factors.” 

For  this  section,  please  provide  a  response  to  all  questions. 

This  section  allows  you  to  describe  certain  job  factors  related  to  your  work 
that  occur  on  an  approximately  daily  basis. 

In  Part  III  of  the  Survey,  you  will  have  a  chance  to  tell  us  about  the  work 
that  you  do  less  often,  like  seasonal  work. 

Turn  to  Page  7. 

Part  II  is  called  “Your  Body’s  Response  to  Work  Demands.” 

This  section  enables  you  to  describe  how  your  body  has  reacted  in  the  past 
to  physical  job  demands. 

For  example,  describing  whether  you  are  comfortable  or  experience  fatigue 
or  discomfort,  is  one  of  the  purposes  of  this  part  of  the  survey. 

Again,  we  will  making  conclusions  about  the  entire  shop  based  on  how  all 
of  you  respond  to  the  Survey  questions. 

We  do  not  intend  to  focus  on  any  one  individual. 
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Turn  to  Page  9. 


Part  III  is  called  “Work  Content.” 

This  section  allows  you  to  list  the  tasks  you  perform  in  your  work  and  how 
often  you  do  them. 

You  will  be  able  to  tell  us  which  types  of  tasks  you  do  and  approximately 
how  often  you  do  the  tasks  over  a  given  period  of  time. 

We  will  use  this  information  to  determine: 

•  what  the  typical/routine  tasks  are  for  your  shop;  and 

•  the  variety  of  tasks  that  are  done  by  your  shop,  even  if  they  are  not  done 
very  often. 

Turn  to  page  1 1 . 

Part  IV  is  called  “Process  Improvement  Opportunities.” 

The  purpose  of  this  section  is  to  identify  the  tasks  that  you  think  place  the 
greatest  demands  on  your  body. 

For  this  section  consider  your  routine,  non-routine,  and  seasonal  tasks  and 
describe  the  tasks  that  you  think  are  a  problem. 

We  will  need  to  know  this  information  in  order  to  help  the  Ergonomics 
Working  Group  decide  which  tasks  may  be  good  candidates  for 
improvement. 
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(START  THE  SURVEY  PROCESS) 

Turn  back  to  Page  1  and  begin. 

We  expect  that  it  will  take  you  about  30  minutes  to  complete  the  Survey. 
When  you  are  finished  with  the  entire  survey,  please  turn  it  in  to  me. 

Thank  you  again  for  your  participation. 

(END  OF  INSTRUCTIONS) 
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APPENDIX  C 


Job  Requirements  and  Physical  Demands 
Survey 


Scoring  Sheets 
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SCORING  SHEET  1 

Although  there  are  many  ways  to  score  the  survey,  we  recommend  that  you  work  through  one  survey  at  a  time, 
completing  the  parts  as  indicated.  Make  sure  your  tally  marks  are  small  enough  so  you  have  room  for  the  entire 
shop. 


Part  I  -  Job  Factors 

A  -  Risk  Factor  Ratings  (Questions  1  -  38) 


Step  1 


responses  in  the  2-4  hour  column  and  in 
the  4-8  hour  column.  If  that  number 
exceeds  the  criteria  number  in  the  box 
in  the  upper  right,  make  one  tally 
mark.  Place  only  one  mark  per  survey  in 
each  box.  Write  the  total  of  die  tallies  in 
the  Total  box. 


Shoulder/Neck  Tally  Box  [_2_ 
Questions  1-7 


HandAVrist/Arm  Tally  Box  [_4^ 
Questions  8-21 


Back/Torso  Tally  Box 
Questions  22-30 


Legs/Feet  Tally  Box 

Questions  31-34 


Head/Eyes  Tally  Box 
Questions  35-38 


Step  2 


Step  3 


Step  4 


Divide  the  Total  tallies 
by  the  number  of  surveys 
from  one  shop. 

Multiply  tiiat  number 
by  100  to  get  the 
percentage. 

Write  the  Risk  Factor  Rating 
(Low,  Med,  High)  in  the  box 
for  each  body  part  using  the 
scale  below. 

Low  Med  High 

^0%  .31-60%  61+% 

number  of 
surveys 

-  =  xl00  = 

=  % 

A.1  Shoulder/Neck 

Risk  Factor  Rating 

A.  2  Hand/Wrist/Arm 
Risk  Factor  Rating 


number  of 
surveys 


xl00  = 


A.3  Back/Torso 
Risk  Factor  Rating 


number  of 
surveys 


xlOO^ 


A.4  Legs/Feet 
Risk  Factor  Rating 


number  of 
surveys 


xl00  = 


A.5  Head/Eyes 
Risk  Factor  Rating 


number  of 
surveys 


xl00  = 
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SCORING  SHEET 
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Part  I  -  Job  Factors 
C  -  Physical  Effort  Score  (Question  45) 


Step  1  Step  2  Step  3 


Write  the  numeric  score  (6-20)  for  each 
survey  in  the  tally  box.  Add  the 
numbers  and  write  the  total  in  the  total 
box. 

Divide  that  total  by  the 
number  of  surveys. 

Write  the  average  in  the  Physical  Effort 
box. 

Tally  Box 

number 

C.  Physical  Effort  Factor 
Score 

of  surveys 

Total 


SCORING  SHEET 
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Part  II  -  The  Body's  Response  to  Work  Demands 
D  -  Discomfort  Rating  (Questions  46  -  60) 


Step  1 


For  each  body  pmt,  look  at  the  responses  to 
the  second  and  third  questions  (47  &  48, 
50&5I,  53&54,  56&37,  59&60);  If 

participants  have  answered  them,  then  look  at 
the  Criteria  Table.  If  the  combination  of 
answers  fits  one  of  the  categories,  then  make 
a  tally  mark  in  the  tally  box  for  each  body 
part.  For  example:  if  47  is  '’weekly"  and  48  is 
"moderate"  then  make  a  tally  mark.  Count 
and  put  total  in  Total  box. 


Step  2 


Step  3 


Daily 


Weekly 


Monthly 


Shoulder/Neck  Tally  Box 
Question  46-48 


Hand/Wrist  Arm  Tally  Box 
Question  49-5 1 


Back/Torso  Tally  Box 
Question  52-54 


Legs/Feet  Tally  Box 
Question  55-57 


Head/Eyes  Tally  Box 
Question  58-60 


Divide  the  total 

Multiply  that  number 

tallies  by  the 

by  100  to  get  the 

number  of  surveys 
from  one  shop. 

percentage. 

Step  4 


Write  the  Discomfort  Rating 
(Low,  Med,  High)  in  the  box 
for  each  body  part  using  the 
scale  below. 


Low  Med  Hish 


<30%  31-60%"  61+% 


Criteria  Table 


number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


D.l  Shoulder/Neck 
Discomfort  Rating 


D.2  HandAVrist/Arm 
Discomfort  Rating 


D.3  Back/Torso 
Discomfort  Rating 


D.4  Legs/Feet 
Discomfort  Rating 


D.5  Head/Eyes 
Discomfort  Rating 
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SCORING  SHEET 


Part  II  -  The  Body’s  Response 
E  -  General  Questions  (Questions  61  -  65) 

Step  1  Step  2 


Look  at  question  61  and  tally  only  Write  the  total  in  the  Health  Care  Provider  Visit  score  box. 
the  "yes"  answers  in  the  tally  box 
for  that  question.  Count  and  write 

the  total  in  the  total  box. _ 

Question  61  Tally  Box  E.  1  Health  Care  Provider 

Visit  Score 


Step  1  Step  2  Step  3  Step  4 


Look  at  each  question  and  tally  Divide  the  total  tallies  Multiply  that  number  Write  the  shop  percentege  in 
only  the  "yes"  answers  in  the  tally  for  that  question  by  the  by  100  to  get  the  the  box  provided., 
box  for  that  question.  Count  and  number  of  surveys.  percentage, 
write  the  total  m  the  Total  box. 


Question  62  Tally  Box 

E.2  Recovery  Time  Score 

number  of 

surveys 

•  = 

X  100  = 

% 

Total 

Question  63  Tally  Box 

E.3  Activity  Interruption 

Score 

number  of 

surveys 

Total 

“ - - 

_x  100  = 

% 

Question  64  Tally  Box 

E.4  Previous  Diagnosis 

number  of 

surveys 

_x  100  = 

% 

Total 

Question  65  Tally  Box 

E.5  Contributing  Factors 

Score 

number  of 

surveys 

_ 

_ x  100  = 

% 

Total 

SCORING  SHEET 
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Part  III 

F  -  Work  Content  (Items  66-122) 


Step  1 


Step  2 


Step  3 


Step  4 


Step  5 


In  the  spaciS  ibelow,  list  item 

For  edch  Routine  Type  of 

Divide  the  total 

Multiply 

that 

Write  in 

the 

number(s)  and  corresponding 

Work,  tally  the  number  of 

tallies  for  each 

number  by 

100 

shop  percentile 

1ype(s)  of  work  that  are 
performed  on  a  “Routine” 
basis. 

tesjjonses.  Count  and  write 
the  total  in  the  total  box. 

type  of  work  by 
the  number  of 
surveys. 

to  get 
percentage. 

the 

in  the 
provided. 

box 

item 

# 


Type  of  Work 


Tally  Box 

Total 

F 

-  =  X 100 

Tally  Box 

Total 

F 

H-  =  X  100 

Tally  Box 

Total 

F 

=  X 100 

Tally  Box 

Total 

F 

-  =  X 100 

Tally  Box 

Total 

F 

H-  =  X  100 

Tally  Box 

Total 

F 

-  =  X 100 

Tally  Box 

Total 

F 

H-  =  X 100 

Tally  Box 

Total 

F 

-  =  X 100 

Tally  Box 

Total 

F 

-  =  X 100 
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APPENDIX  D 


Job  Requirements  and  Physical  Demands 
Survey 


Summary  Report 
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1 


SUMMARY  REPORT 


ERPA  Status: 

Date:  ; 

Date: 

Workplace  Identifier: 

Base: 

Organization: 

Workplace: 

Bldg./Location: 

Room/ Area 

AFSC: 

Civiliein  Job  Series: 

Shop  Supervisor: 

Duty  Phone: 

Office  Symbol: 

Write  in  the  Risk  Factor  Ra  W^rite  m  the  Discomfoit  Rati^  Look  at  the  "Ranking  Matrix"  below 

for  Part  L  (questions  1t3  8,  Scoring  for  Part  11,  (questions  46-60,  and  enter  the  Priority  Score  in  it's 
Sheet  pg.l)  Scoring  Sheet pg.3)  coifesponding  box. 


Shoulder/Neck  = 


HandsAVrist/Arms  = 


Back/Torso  = 


Legs/Feet  = 


Head/Eye  = 


Ranking 

Matrix 


Ranking  Matrix  for 
Priority  Score 


Risk  Factor  High 


Risk  Factor 
Medium 


Risk  Factor  Low 


Discomfort  High  Discomfort  Medium  Discomfort  Low 
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Summary  Report 


1  Step  4 

B.  Enter  Organizational  Factor 
Rating:  (Questions  39-44,  Scoring  Sheet  ^ 
pg.2) 

Comments: 

.•••'  :  '  ■■■  /.I  .,  ^  % 

Step  5 


C.  Enter  Physical  Effort  Factor 

Score:  (Question  45,  Scoring  Sheet 
Pg-2) 

Comments: 

Step  6 

E.  Enter  the  score  for  each  of  the  General  Questions:  (Questions  61-65,  Scoring  Sheet  pg.  4) 

E.l  Health  Care  Provider  Score 

Comments: 

E.2  Recovery  Time  Score 

Comments: 

% 

E.3  Activity  Interruption  Score 

Comments: 

% 

E.4  Previous  Diagnosis  Score 

Comments: 

% 

E.5  Contributing  Factors  Score  Comments: 


% 


Step  7 


F.  List  below  each  of  the  routine  types  of  work  which  had  shop  percentage  scores  over  20%.  (Items  66-122,  scoring 
sheet  page  5) 


Type  of  Work 


% 


Type  of  Work 


% 


Summary  Report 
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Step  8  1 

Review  Part  I¥  (Questions  1-3)  to 
identify  tasks,  tools,  equipment,  etc., 
that  employees  listed  as  potential 
concerns.  Comment  as  appropriate. 

Comments: 

Review  Part  IV  (Question  4)  to 
identify  potential  improvement 
opportunities.  Comment  as 
appropriate. 

Comments: 

1  Step  9  1 

Injury/Illness  Data:  Review  the 
injuiy/illness  history  from  this  shop. 
Attach  information  and  comment  as 
appropriate.  , 

Comments: 

Step  10 

Conclusions  /  Recommendations  Summary 


Shop  Status  Recommendations  for  follow-up: 
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Job  Requirements  and  Physical  Demands 
Survey 


Scientific  Basis  for  the  Job  Requirements  and  Physical  Demands  Survey 
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1.0  OVERVIEW  OF  THE  JOB  REQUIREMENTS  AND  PHYSICAL  DEMANDS 
SURVEY  DEVELOPMENT 


The  purpose  of  this  document  is  to  provide  the  Survey  development  criteria,  development 
rationale,  and  the  rationale  and  results  of  all  testing  and  validation  procedures  performed  during 
the  Survey  development.  Section  1.0  is  a  brief  overview  of  the  entire  project.  Section  2.0 
contains  detailed  information  regarding  the  Survey  development,  and  Section  3.0  contains 
detailed  information  regarding  the  Survey  testing  and  validation  procedures.  The  attachments 
contain  documentation  for  each  question  used  in  the  Survey,  statistical  validation  summaries,  and 
raw  statistical  data. 

1.1  Development  Criteria 

One  of  the  Air  Force’s  primary  objectives  was  to  develop  a  written  tool  that  can  be  easily 
administered  to  work  area  employees  by  Public  Health  technicians.  Other  specific  design 
criteria,  established  by  the  Air  Force,  are  listed  in  Table  1.1. 

Table  1.1  Job  Requirements  and  Physical  Demands  Survey  Development  Criteria 


•  The  Survey  is  designed  to  be  administered  to  an  assembled  group  of  work  area  employees 
within  one  hour. 

•  The  Survey  is  designed  to  enable  a  Public  Health  technician  to  analyze  the  data  for  25 
work  area  employees  within  four  consecutive  hours. 

•  The  Survey  provides  a  means  for  employees  to  identify  specific  work  processes, 
activities,  and  tasks  which  they  believe  are  related  to  their  reported  musculoskeletal 
discomfort  and/or  work-related  muskuloskeletal  disorders  (WMDs). 

•  Results  of  the  Survey  will  help  the  base  Ergonomics  Working  Group  (EWG)  determine  if 
a  Potential  Ergonomic  Problem  Area  (PEP A)  should  be  classified  as  an  Ergonomic 
Problem  Area  (EPRA).  An  EPRA  is  a  work  area  where  an  association  can  be  shown 
between  ergonomic  risk  factors,  employee-reported  musculoskeletal  discomfort,  and 
employee-reported,  medically  confirmed  WMDs  (if  applicable). 

•  Results  of  the  Survey  will  prioritize  EPRA-classified  work  areas  for  “task  specific” 
analyses  and/or  problem-solving  efforts. 

•  Results  of  the  Survey  provide  an  indication  of  and  the  relative  importance  of  ergonomic, 
psychosocial,  and  individual  factors  that  may  be  present  in  the  work  area. 

•  Data  from  the  Survey  allows  calculation  of  employee-reported  discomfort  prevalence 
rates. 
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In  addition,  while  the  primary  purpose  of  the  Survey  is  not  to  judge  the  effectiveness  of  the  Air 
Force  injury  and  illness  reporting  system,  the  Survey  data  from  a  particular  work  area  should 
enable  the  Public  Health  technician  and/or  the  EWG  to  determine  whether  it  is  likely  that 
employees  are  under-reporting  their  musculoskeletal  discomfort  or  symptoms  of  WMDs. 

1.2  Development  Process 

The  Survey  design  is  the  result  of  an  iterative  development,  testing,  and  validation  process  that 
enlisted  and  benefited  from  the  support  and  cooperation  of  Air  Force  personnel  at  several  Air 
Force  installations: 

•  Armstrong  Laboratory,  Brooks  AFB,  TX; 

•  Cape  Cod  AS,  MA; 

•  Malmstrom  AFB,  MT; 

•  Patrick  AFB,  FL;  and 

•  Peterson  AFB,  CO. 

1.2.1  Development  of  the  Initial  Survey 

The  development  process  began  with  a  review  of  the  scientific  literature.  The  purpose  of  the 
review  was  to  identify  other  screening  tools  or  features  of  other  screening  tools  that  could  be 
used  to  satisfy  the  criteria  established  by  the  Air  Force.  The  literature  review  revealed  that  there 
is  a  lack  of  established  and  validated  employee  survey  tools  for  prioritizing  ergonomic  hazards  in 
the  workplace.  (For  more  information  see  Section  2,  pp.  2-1).  When  possible,  individual 
questions  were  extracted  from  surveys  or  questionnaires  reported  in  peer  reviewed  journals. 
Questions  were  also  extracted  either  from  widely  used  surveys  or  created  by  extrapolating  from 
established  ergonomic  risk  factors.  This  course  of  actions  was  taken  to  maximize  the  use  of 
existing,  albeit  limited  information. 

The  process  continued  with  site  visits  to  selected  USAF  Space  Command  installations:  Cape 
Cod  AS,  MA;  Malmstrom  AFB,  MT;  and  Patrick  AFB,  FL.  The  ergonomists  video-taped  jobs  in 
PEP  A  and  non-PEPA  shops.  The  members  elicited  input  from  Public  Health  offices  and 
Bioenvironmental  Engineering  Services  to  ensure  familiarity  with  the  type  of  work  being 
performed  in  the  shops.  The  purpose  of  the  site  visits  was  to  ensure  that  the  final  Survey  tool 
reflected  the  types  and  varieties  of  work  situations  foimd  throughout  the  Air  Force. 

The  Survey  incorporated  the  results  of  the  literature  review  and  site  visits,  criteria  established  by 
the  Air  Force,  and  a  series  of  discussions  with  Air  Force-designated  technical  advisors.  An 
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iterative  approach  was  used  in  order  to  incorporate  ideas  from  all  Survey  contributors.  Prior  to 
conducting  the  reproducibility  testing,  seven  different  versions  of  the  Survey  had  been 
developed. 

1.2.2  Testing  and  Validation 

The  purpose  of  the  testing  and  validation  process  was  to  establish  the  strengths  and  limitations  of 
the  initial  Survey  and  to  identify  the  need  for  changes  based  on  quantitative  information.  The 
testing  and  validation  process  was  conducted  in  three  distinct  phases:  usability  testing, 
reproducibility  testing,  and  validity  testing.  An  overview  is  provided  below.  (For  a  detailed 
discussion  of  the  testing  and  validation  process  and  results,  refer  to  Section  3). 

Usability  testing  was  performed  to  ensure  Public  Health  technicians  would  be  able  to  use  the 
Survey  as  intended,  that  the  Survey  met  the  design  criteria,  and  that  the  questions  were 
imderstandable.  The  testing  was  conducted  at  Malmstrom  AFB  and  focused  on  the  Survey 
administration  process  (e.g.,  adherence  to  completion  time  criteria),  the  Survey  questions  (e.g., 
understandable,  applicable,  etc.),  and  the  scoring  procedures  (e.g.,  adherence  to  completion  time 
criteria,  ease  of  use,  etc.).  Input  from  the  test  group  at  Malmstrom  AFB  resulted  in  significant 
improvements  to  ease  the  use  and  efficiency  of  the  Survey  administration  and  scoring  process. 

Reproducibility  testing  was  performed  to  determine  how  consistently  the  Survey  yielded  the 
same  results.  In  other  words,  the  testing  was  done  to  see  if  employees  responded  the  same  way 
to  a  question  when  the  Survey  was  administered  at  two  different  times.  Test/re-test 
reproducibility  was  examined  for  the  Survey  since  it  is  a  self-reporting  (employee)  tool.  A  two- 
week  test/re-test  evaluation  was  conducted  at  Peterson  AFB.  This  time  period  was  selected  to 
ensiue  sufficient  delay  so  that  participants  would  not  remember  their  responses  while 
maintaining  a  short  enough  interval  that  the  participant’s  job  demands  and  discomfort  would 
remain  constant. 

Validity  testing  was  conducted  to  measure  how  closely  the  results  (e.g..  Ergonomic  Priority 
Ranking  for  several  work  areas)  from  an  experienced  ergonomist  matched  those  which  were 
generated  from  administering  the  Survey.  The  similarity  of  rankings  determined  how  closely  the 
two  measures  agreed  with  each  other.  An  experienced  Ergonomist  visited  31  work  areas  prior  to 
administration  of  the  Survey:  five  at  Cape  Cod  AS  and  28  at  Patrick  AFB.  Several  measures 
were  taken  to  prevent  the  Ergonomist  from  biasing  the  Survey  responses.  This  included  the 
requirement  that  the  Survey  could  not  be  administered  to  a  work  area  visited  by  the  Ergonomist 
until  two  weeks  had  passed.  The  two-week  time  period  was  established  to  minimize  the 
potential  that  employees  would  respond  to  a  Survey  question  based  on  discussions  that  may  have 
occurred  during  the  Ergonomist’s  shop  visits  while  again  maintaining  a  short  enough  interval 
that  the  participants  job  demands  and  discomfort  would  remain  constant.  The  Survey  was 
administered  by  base  personnel  from  Patrick  AFB  and  Peterson  AFB  (in  cooperation  vdth  Cape 
Cod  AS).  Results  from  the  validity  testing  revealed  that  there  was  a  statistically  significant 
correlation  between  the  overall  work  area  Ergonomic  Priority  Rankings  provided  by  the 
Ergonomist  and  those  which  resulted  from  administration  of  the  Survey.  This  means  that. 
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overall,  the  Survey  results  would  be  expected  to  agree  with  the  findings  of  an  experienced 
ergonomist. 
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2.0  PRACTICAL  BASIS  FOR  THE  SURVEY  METHODOLOGY 


This  section  of  the  research  report  contains  detailed  information  related  to  the  development  of  the 
Survey,  including  the  Survey  criteria  and  the  rationale  and  reasoning  used  to  select,  modify, 
and/or  develop  each  question. 

2.1  Literature  Review 

The  initial  step  in  the  Survey  development  process  was  to  perform  a  literature  review  in  order  to 
determine  if  a  survey  screening  tool  was  the  most  appropriate  method  to  use  to  obtain  both 
ergonomic  risk  factor  and  health  surveillance  data.  Another  purpose  of  the  literature  review  was 
to  identify  other  screening  tools  or  features  of  other  screening  tools  that  could  be  used  to  satisfy 
the  criteria  established  by  the  Air  Force.  The  literature  review  revealed  a  wide  range  of  job- 
focused  risk  assessments  requiring  levels  of  ergonomic  knowledge  varying  from  none  to  those 
possessed  by  an  experienced  ergonomist  (Cole,  1995  [1];  Keyserling  et  al,  1993  [2];  Reynolds, 
Drury  &  Broderick,  1994  [3]).  Most  of  the  established  methods  require  at  least  several  days  of 
training  to  complete  properly.  Several  job  specific  screening  methods  have  been  developed  and 
used  to  “survey”  employees  vdth  little  or  no  ergonomic  training  ([1],  [3]).  Surveys  designed  for 
collecting  epidemiological  data  from  employees  have  also  been  developed  (Wiktorian  et  al,  1991 
[4]). 

There  is  one  fundamental  difference  between  an  epidemiological  survey  and  a  screening  survey. 
An  epidemiology  survey  identifies  activities  that  have  historically  been  associated  with  WMDs 
but  does  not  provide  directive  information  about  future  intervention.  A  screening  survey  is 
generally  used  to  target  or  prioritize  jobs  for  intervention. 

The  primary  factors  which  were  considered  when  selecting  and  designing  the  appropriate 
screening  methodology  are: 

1 .  What  level  of  expertise  is  required? 

2.  What  are  the  time  requirements? 

3 .  What  are  the  associated  costs? 

4.  How  invasive  is  the  methodology? 

5 .  How  valid  are  the  scores  obtained? 

Assessment  methods  currently  in  use  were  reviewed  by  The  Joyce  Institute/A  Unit  of  Arthur  D. 
Little,  Inc.  ergonomists  to  identify  advantages  or  features  which  could  be  incorporated  into  the 
Survey  design.  Table  2.1  summarizes  the  advantages  and  disadvantages  of  the  various 
assessment  approaches  based  on  results  of  the  ergonomists’  review. 
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Table  2.1  Advantages  and  Disadvantages  of  Ergonomic  Assessment  Approaches 


Assessment 

Type 

Method 

Advantages 

Disadvantages 

Passive 
surveillance 
(screening) 
for  WMDs 

Records  Review  - 
OSHA  200  log, 
medical  reports, 
nurses  logs, 
workers’ 
compensation 
reports,  insurance 
reports. 

•  Process  can  be  quick 
and  inexpensive  if 
data  is  readily 
available. 

•  Possible  to  prioritize 
action  according  to 
frequency,  severity, 
and/or  cost  of  cases. 

Conclusions  made  from 
the  data  are  typically 
limited  to  identifying 
departments  or  work 
areas  -  not  tasks  in 
which  problems  may 
exist.  Approach  is 
totally  reactive  - 
judgments  are  made 
based  only  on  reported 
injuries.  Other  factors 
may  influence  injury 
reporting  (e.g., 
downsizing)  and  cost 
(e.g.,  case  management). 

Active 
surveillance 
for  WMDs 

Physician-  or 

Health  Care 
Provider- 
sponsored  health 
assessments 

•  Provides  detailed 
baseline  information 
on  individual  and 
group  employee 
health.  Qualified 
health  care  personnel 
perform  evaluations 
(e.g.  Phalens  test, 
Tinels  Sign,  etc.)  to 
identify  symptoms  or 
conditions  that  may 
indicate  the  presence 
or  onset  of  WMDs. 

•  Information  on 
individual  factors 
(e.g.,  previous 
injuries)  can  be 
obtained. 

Assessment  process  is 
expensive  and  time¬ 
intensive.  Does  not 
indicate  potential  source 
of  symptoms  or 
contributing  job  factors. 
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Table  2.1  Advantages  and  Disadvantages  of  Ergonomic  Assessment  Methods  (Contd.) 


Assessment  Type 

Method 

Advantages 

Disadvantages 

Active 

surveillance  for 
risk  factors 

Questionnaires  - 
surveys  or 
interviews 

•  Questionnaires  are 
quick  and 
inexpensive  to 
administer  and  are 
non-invasive.  Few 
technical  skills  are 
required  for  survey 
administration.  They 
can  also  provide 
some  indication  of 
potential  sources 
(jobs,  tasks,  etc.)  of 
employee-reported 
discomfort. 

Provides  only  general 
information  on  exposure  to 
ergonomic  risk  factors.  Due 
to  the  subjective  nature  of 
the  data  collection  process, 
results  may  have  lower 
validity  than  those  from 
other  methods. 

Active 

surveillance  for 
risk  factors 
(cont.) 

Observational 
Techniques- 
checklists,  task 
analyses 

i 

•  Process  requires 
only  moderate  time 
and  cost  to  perform 
and  has  a  low  level  of 
invasiveness. 

•  Techniques  can 
provide  detailed 
information  on  risk 
factor  exposure,  and 
identification  of  root 
causes  and  potential 
control  options. 

•  Analysis  results 
may  be  used  to 
prioritize  action  on 
specific  tasks. 

Moderate  level  of  technical 
skill  is  required.  Results 
provide  moderate  detail  and 
validity.  Conclusions  on 
priority  for  change  are  based 
on  the  training  and 
observational  skills  of  the 
technician. 

Active 

surveillance  for 
risk  factors 
(cont.) 

Direct 

Measurements- 
EMG,  electronic 
sensors, 
goniometers 

•  Provides  higher 
level  of  detail  and 
precision.  Data  can 
be  used  to  assess 
potential  risk  of 
exposure  when 
standards  are 
available  (e.g., 
vibration  exposure). 
Methods  provide 

Process  is  often  costly,  time 
intensive,  and  requires  a  high 
level  of  technical  skill  to 
perform  accurate 
measurements.  Direct 
measurement  can  be  highly 
invasive.  Costs  of  obtaining 
data  may  outweigh  the  value 
as  a  problem-solving  tool 
due  to  the  lack  of  availability 
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Assessment  Type 

Method 

Advantages 

Disadvantages 

standardized  means 
for  measuring 
reduction  in  exposure 
after  improvements 
are  made. 

of  validated  exposure  data 
(e.g.,  grip  force). 

The  literature  review  indicated  that  a  questionnaire/survey  approach  was  the  most  appropriate 
method  for  obtaining  both  risk  factor  and  health  surveillance  data. 

2.2  Survey  Design 

The  Survey  was  designed  to  accomplish  the  following  specific  objectives  listed  below: 

•  The  Survey  can  be  administered  to  an  assembled  group  of  work  area  employees  within 
one  hour. 

•  The  Survey  design  enables  the  Public  Health  technician  to  analyze  the  data  for  25  work 
area  employees  within  four  consecutive  hours. 

•  The  Survey  provides  a  means  for  employees  to  identify  specific  work  processes, 
activities,  and  tasks  which  they  believe  are  related  to  their  reported  musculoskeletal 
discomfort  and/or  WMD. 

•  Results  of  the  Survey  will  help  the  base  EWG  members  determine  if  a  PEPA  should  be 
classified  as  an  EPRA. 

•  Results  of  the  Survey  will  be  used  to  prioritize  EPRA-classified  work  areas  for  “task 
specific”  analyses  and/or  problem-solving  efforts. 

•  Results  of  the  Survey  provide  an  indication  of  and  the  relative  importance  of  ergonomic, 
psychosocial,  and  individual  factors  which  may  be  present  in  the  work  area. 

•  Data  from  the  Survey  allows  calculation  of  employee-reported  discomfort  prevalence 
rates. 

The  above  objectives  are  consistent  with  the  requirement  to  provide  a  quick  and  effective 
screening  tool  which  prioritizes  shops,  identifies  the  potential  source(s)  of  exposure  to  ergonomic 
risk  factors,  and  suggests  strategies  for  follow-up. 

Another  primary  design  objective  was  to  develop  one  survey  which  would  be  applicable  to  each 
of  four  work  area  types  found  throughout  the  Air  Force.  The  work  areas  are: 


2-4 

54 


Maintenance/Inspection; 


•  Assembly  Line; 

•  Warehouse;  and 

•  Administrative. 

The  Survey  also  had  to  obtain  data  which  related  to  both  the  employee’s  physical  experience 
(e.g.,  comfort,  discomfort,  etc.)  with  the  job,  as  well  as  the  overall  exposure  to  ergonomic  risk 
factors.  Priorities  were  to  be  established  based  on  both  types  of  information.  In  addition,  since 
the  results  were  also  intended  to  be  used  to  identify  opportunities  for  problem  solving,  the 
Survey  was  designed  to  identify  common  tasks  (e.g.  what  do  the  employees  do  on  a  routine 
basis)  as  well  as  tasks  or  activities  that  employees  believe  may  be  a  source  of  concern.  As  a 
result,  the  Survey  is  comprised  of  four  sections: 


Part  I: 

Description  of  Work 

Part  II: 

Your  Body’s  Response  to  Work  Demands 

Part  III: 

Work  Content 

Part  IV: 

Process  Improvement  Opportunities. 

2.2.1  Cover  Page 

The  cover  page  enables  employees  to  identify  the  workplace  (shop)  and  location  of  work  as  well 
as  to  specify  selected  employment-related  demographics. 


Information  collected  on  the  cover  page  (Figure  2.1)  is  expected  to  be  used  only  for  record 
keeping.  Specifically,  the  information  will  not  be  used  to  identify  individuals  within  any  of  the 
shops.  Data  requested  on  demographics  (e.g.,  gender,  work  group,  age  category,  length  of 
service,  etc.)  may  be  used  by  the  Air  Force  in  post  hoc,  installation-wide  analyses. 
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Figure  2.1  Cover  Page 


Job  aud  Physical 

.  Bemauds  Survey 

Date  (YYMMDD) 

Workplace 

IdeutiOer; 

(use  this  space  for  mechanical  imprint) 

Base 

Organization 

Workplace 

Bldg.  No/Location 

Room/Area 

AFSC/Job  Series 


Gender: 

Female  O 

_ \ _ 

Male  O 

Work  Group: 

Civilian  O  Grade: 

Military  O  Rank: 

Age  Category: 

20  and  under  O  21-30  O  31-40  O  over  40  O 

Length  of  service  at  this  base:  less  than  one  year  O 

more  than  one  year  O 

Length  of  time  in  current  shop:  less  than  one  year  O 

more  than  one  year  O 

Have  you  completed  this  questionnaire  before? 

Yes  O  No  O 
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2.2.2  Parti:  Description  of  Work 


Part  I  is  divided  into  three  sections:  Job  Factors,  Organizational  Factors,  and  Physical  Effort. 

2.2.2. 1  Job  Factors 

The  Job  Factors  section  will  provide  Public  Health  information  on  the  extent  employees  may  be 
exposed  to  ergonomic  risk  factors  that  may  contribute  to  WMDs.  The  following  paragraphs 
describe  the  research  findings  associated  with  developing  this  portion  of  the  Survey,  as  well  as 
the  purpose  of  the  questions,  the  rationale  for  the  responses,  and  what  the  results  indicate.  An 
excerpt  from  the  Job  Factors  section  is  included  as  Figure  2.2. 

2.2.2.1.1  Research  Findings 

The  section  examines  the  prevalence  of  exposure  to  ergonomic  risk  factors  within  a  shop.  The 
categories  for  duration  of  exposure  (never,  less  than  2  hours,  2  to  4  hours,  4  to  8  hours)  were 
established  based  on  the  OSHA  draft  standard.  Ergonomic  risk  factors  to  which  employees  were 
exposed  an  average  of  less  than  2  hours  per  day  have  a  lower  priority  than  more  prevalent 
factors.  The  selection  of  individual  questions  for  inclusion  in  the  risk  factor  section  of  the 
Survey  was  based  on  the  steps  listed  below. 

1 .  Review  of  literature  for  existing  questions. 

2.  Review  of  literature  for  established  risk  factors. 

3.  Selection  of  questions  and  risk  factors  represented  in  Air  Force  tasks. 

4.  Formulation  of  first-person  statements  based  on  existing  questions  and  established  risk 
factors. 

5 .  U sability  testing  conducted  at  Malmstrom  AFB . 

6.  Revisions  to  question  wording  and  selection  based  on  user  feedback. 

7.  Reproducibility  testing  at  Peterson  AFB. 

8.  Revision  based  on  results  of  reproducibility  testing. 

A  review  of  the  literature  indicated  that  there  is  a  lack  of  established  and  validated  employee 
survey  tools  for  prioritizing  ergonomic  hazards  in  the  work  place.  The  lack  of  validated 
ergonomic  surveys  has  been  noted  by  other  researchers  (Buckle,  1995  [5];  Burdorf,  1992  [6]; 
Kilbom,  1994  [7];  Wiktorian  et  al,  1993  [4];  Baron  et  al,  1996  [8]).  The  lack  of  validated 
assessment  surveys  is  not  limited  to  the  field  of  ergonomics;  it  has  also  been  noted  in  work 
history  (Bond  et  al,  1988  [9])  and  physical  activity  measures  (Washburn  and  Montoye,  1986 
[10]). 
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Figure  2.2  Job  Factors 


A.  DESCRIPTION  OF  WORK 


Figure  A. 


1- 

Figure  B, 


SHOULDER /NECK 


^  V 


I  work  with  my  hands  at  or  above  chest  level.  (Figure  A,) . 

o 

o 

o 

o 

To  get  to  or  to  do  my  work,  I  must  lay  on  my  back  or  side  and 
work  with  my  arms  up . 

o 

o 

o 

o 

I  must  hold  or  carry  materials  (or  large  stacks  of  files)  during 
the  course  of  my  work . 

o 

o 

o 

o 

I  force  or  yank  components  or  work  objects  in  order  to  complete 
a  task . 

o 

o 

o 

o 

I  reach  or  hold  my  arms  in  front  of  or  behind  my  body  (e.g., 
using  a  keyboard,  filing,  handling  parts,  performing  inspection 
tasks,  pushing  or  pulling  carts,  etc.).  (Figures  B,)  . 

o 

o 

o 

o 

6.  My  neck  is  tipped  forward  or  backward  when  I  work.  {Figure 
C.) . 


O 


o 


7.  I  cradle  a  phone  or  other  device  between  my  neck  and  shoulder. 

(Figure  D.) .  q 


o  o 


o 
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The  only  two  employee  survey  tools  identified  in  the  literature  which  have  undergone  any 
reproducibility  research  (Cole  &  Rosa,  1994  [11];  [8])  focus  on  epidemiological  descriptions  of 
work  place  stressors  or  activities  without  prioritizing  or  classifying  the  resulting  risks.  Other 
survey  tools  have  been  reported  in  the  literature  without  accompanying  validation  research.  The 
reproducibility  and/or  validity  of  several  observational  analysis  questioimaires  has  been  reported 
(Kemmlert,  1994  [12];  Lifshitz  &  Armstrong,  1986  [13];  McAtamney  &  Corlett,  1993  [14]). 

Where  possible,  questions  were  extracted  from  survey  or  questionnaire  tools  reported  in  peer 
reviewed  journals.  In  other  cases,  the  questions  were  extracted  from  either  widely  used  surveys 
(Steelcase,  imdated  [15])  or  extrapolated  fi-om  established  risk  factors  (e.g.,  stressful  postures, 
excessive  force,  etc.).  Questions  were  selected  and  modified  to  describe  risk  factors  in  terms 
associated  with  the  tasks  which  employees  perform  [7].  A  complete  list  of  risk  factor  questions 
and  the  research  literature  supporting  their  inclusion  as  Job  Factor  Questions  is  presented  in 
Attachment  1  of  this  Appendix.  Each  of  the  questions  was  altered  slightly  such  that  it  would  be 
read  in  first  person.  Responses  to  all  questions  were  altered  to  correspond  to  the  time  categories 
selected.  Since  the  original  risk  factor  verbiage  was  essentially  retained,  these  slight  alterations 
are  not  expected  to  impact  past  reported  reproducibility  and/or  validity.  Several  questions  were 
altered  more  significantly.  For  example,  question  1,  “I  work  with  my  hands  at  or  above  chest 
level.”  was  modified  from  the  original  “Is  an  elbow  used  at  or  above  mid-torso  level?”  to 
achieve  a  more  direct  expression  of  the  risk  factor.  Since  “chest  level”  is  easier  to  identify  by  the 
non-specialist  than  “mid-torso  level”,  the  change  was  expected  to  maintain  if  not  improve 
question  reproducibility  (i.e.,  remove  the  need  for  user  interpretation).  The  risk  factor  basis  for 
the  question  is  retained.  Therefore,  while  some  of  the  alterations  may  appear  substantial,  the 
changes  are  not  expected  to  reduce  past  reported  reproducibility  and/or  validity.  A  pilot 
investigation  of  test/retest  reproducibility  was  performed  on  these  revised  questions  to  verify  that 
reproducibility  remained  consistent  or  improved. 

2.2.2.1.2  Questions  (Q1-Q38) 

In  the  Job  Factors  section,  the  employee  is  asked  to  respond  to  a  series  of  questions  which  relate 
to  the  variety  of  physical  demands  in  work  activities.  The  questions  have  been  grouped  into  five 
“body  zones”:  shoulder/neck  (Q1-Q7),  hand/wrist/arm  (Q8-Q21),  back/torso  (Q22-Q30), 
legs/feet  (Q31-33),  and  head/eyes  (Q35-Q38).  The  questions  are  representative  of  the  types  of 
ergonomic  risk  factors  that  are  most  likely  to  be  fovmd  in  Air  Force  work  situations. 

The  specific  questions  that  are  included  were  designed  to  ensure  that  each  general  risk  factor  type 
discussed  in  the  scientific  literature  (e.g.,  posture,  force,  repetition,  etc.)  were  reflected.  The 
questions  were  also  designed  such  that  they  would  be  applicable  to  each  of  the  four  work  area 
types  found  throughout  the  Air  Force:  Administrative,  Assembly,  Maintenance/Inspection,  and 
Warehouse. 

Each  section  has  a  different  number  of  questions.  For  example,  there  are  14  questions  for 
hand/wrist/arm  and  four  questions  for  legs/feet.  The  number  of  questions  in  each  section 
generally  reflects  the  present  state  of  ergonomic  research  and  knowledge  about  risk  factors.  In 
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other  words,  hand/wrist/arm  ergonomic  research  has  provided  greater  insight  into  the  potential 
causes  of  hand/wrist/arm  WMDs  than  research  which  has  been  conducted  for  the  legs/feet. 

2.2.2.1.3  Responses 

For  each  question,  the  employee  estimates  the  approximate  amoimt  of  time  that  their  work 
exposes  them  to  that  job  factor  (e.g.,  I  work  with  my  hands  at  or  above  chest  level).  The  choices 
are:  0-2  hours,  2-4  hours,  4-8  hours,  or  never/NA.  The  first  three  response  choices  were  selected 
to  remain  consistent,  in  concept  with  the  1995  OSHA  Draft  Ergonomics  Standard  [16]  and  the 
response  categories  proposed  in  the  American  National  Standards  Institute  (ANSI)  National 
Safety  Council  Draft  Standard  Z-365  (ANSI  Z-365)  (ANSI,  1995  [17]).  “Never”  was  added  in 
response  to  feedback  obtained  during  the  alpha  and  beta  testing  phases  of  the  development 
process.  For  example,  employees  who  were  never  exposed  to  a  particular  Job  Factor  commented 
that  responding  in  the  “0-2  hours”  category  seemed  like  an  overstatement.  Furthermore, 
employees  who  worked  or  were  exposed  to  a  Job  Factor  for  1-1/2  hours  commented  that  since 
they  were  being  grouped  with  employees  who  were  exposed  for  “0”  hours,  their  own  exposure 
was  discoimted. 

2.2.2.1.4  What  the  Section  Indicates 

Responses  averaged  across  the  shop  indicate  the  extent  to  which  employees  may  be  exposed  to 
ergonomic  risk  factors  that  may  contribute  to  WMDs.  In  addition  grouping  the  questions  by 
body  zones  also  helps  identify  the  body  zone(s)  which  may  be  exposed  to  the  greatest  extent  and 
allows  for  comparison  to  responses  in  the  discomfort  section,  which  are  also  grouped  by  body 
zone.  This  will  help  Public  Health  and  the  EWG  establish  targets  for  effective  problem  solving 
strategies  for  shops  that  are  upgraded  to  EPRA  status. 

The  “Risk  Factor  Rating”  for  each  body  zone  is  used  in  the  Survey  Priority  Rank  calculation  for 
the  shop. 

2.2.2.2  Organizational  Factors 

The  following  paragraphs  describe  the  research  findings  associated  with  developing  this  portion 
of  the  Survey,  as  well  as  the  purpose  of  the  questions,  the  rationale  for  the  responses,  and  what 
the  results  indicate.  The  Organizational  Factors  section  is  included  as  Figure  2.3. 
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Figure  2.3  Organizational  Factors 


B.  ORGANIZATIONAL  FACTORS 
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39. 1  often  feel  unclear  on  what  the  scope  and 

responsibilities  of  my  job  are . 

o 

o 

o 

o 

o 

40.1  often  feel  that  I  have  too  heavy  of  a 

workload,  one  that  I  could  not  possibly 
finish  during  an  ordinary  workday . 

o 

o 

o 

o 

o 

41. 1  often  feel  that  I  will  not  be  able  to  satisfy 

the  conflicting  demands  of  various  people 
around  me . . . 

o 

o 

o 

o 

o 

42.1  often  find  myself  unable  to  get 

information  needed  to  carry  out  my  job . 

43.1  often  do  not  know  what  my  supervisor 

o 

o 

o 

o 

o 

thinks  of  me,  how  he/she  evaluates  my 
performance . 

o 

o 

o 

o 

o 

44. 1  often  think  that  the  amount  of  work  I  have 

to  do  may  interfere  with  how  well  it’s  done. 

o 

o 

o 

o 

o 

2.2.2.2.1  Research  Findings 

Questions  related  to  job  stress  (overload,  role  ambiguity,  recognition,  job  suitability)  are  included 
in  the  stirvey  to  reflect  the  fact  that  the  employee  is  part  of  an  environment  which  creates  both 
physical  and  organizational  demands.  These  issues  can  be  important  to  the  Air  Force  since 
studies  have  shown  that  organizations  that  ignore  these  issues  may  experience  decreases  in 
productivity,  as  well  as  increases  in  injuries/illnesses.  Workers’  Compensation  claims,  and 
absenteeism.  In  one  study  (Bigos  et  al,  1991  [18]),  which  involved  3,020  workers  over  a  four 
year  period,  279  back  injuries  were  reported.  Other  than  prior  injuries,  organizational  factors 
were  the  most  significant  predictors  of  claims.  The  researchers  concluded  that  report  of  (back) 
injury  is  an  event  that  may  be  influenced  by  a  complex  set  of  factors  that  cannot  be  understood 
solely  in  physical  ergonomic  terms.  Each  of  the  questions  included  in  the  Survey  were  taken 
directly  from  a  study  on  organizational  stress  and  its  impact  on  absenteeism  (Khan  et  al,  1964 
[19]).  The  questions  were  used  with  only  minor  modifications  (to  fit  the  Survey  language)  since 
they  already  expressed  the  organizational  issues  in  an  unemotional/balanced  way.  A  five-point 
strongly  disagree  or  strongly  agree  scale  was  chosen  for  the  responses  because  it  is  a  familiar 
format  and  because  five-point  scales  appear  to  provide  the  greatest  utility  (Meister,  1985).  (Since 
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the  questions  were  all  taken  from  the  same  research  source,  a  question-by-question  discussion, 
which  was  presented  in  the  Job  Factors  section  [taken  from  many  research  sources],  is  not 
necessary.) 

As  a  result,  questions  related  to  job  stress  (overload,  role  ambiguity,  recognition,  job  suitability) 
have  been  added  to  the  Survey  to  reflect  the  fact  that  the  employee  is  part  of  an  environment 
which  creates  both  physical  and  organizational  demands. 

1.222.2  Questions  (Q39-Q44) 

There  are  six  questions  that  deal  with  organizational  issues.  Organizational  Factors  are  included 
in  the  Survey  to  provide  Public  Health  with  insight  into  how  employees’  well-being  (e.g.,  reports 
of  discomfort)  may  be  related  to  how  they  perceive  their  worth  to  the  organization. 

2.2.2.2.3  Responses 

The  employee  is  asked  to  respond  to  each  question  using  a  five-point  scale:  strongly  disagree, 
disagree,  neutral,  agree,  and  strongly  disagree. 

2.2.2.2.4  What  the  Section  Indicates 

Responses  averaged  across  the  shop  indicate  the  possible  extent  to  which  Organizational  Factors 
may  be  influencing  the  employees’  attitude  towards  their  work.  For  example,  if  the 
Organizational  Factor  results  for  the  entire  shop  indicate  that  the  majority  of  the  employees 
understand  their  responsibilities,  feel  that  the  work  load  is  reasonable,  feel  that  they  are  able  to 
satisfy  the  demands  of  others,  are  able  to  get  the  information  that  they  need  to  carry  out  their 
jobs,  and  so  on.  Public  Health  may  feel  confident  about  the  results  generated  from  the  Job 
Factors  section.  In  other  words.  Public  Health  would  not  need  to  be  concerned  (as  they  might  be 
if  the  entire  shop  reported  the  presence  of  negative  Organizational  Factors  -  e.g.,  the  employees 
think  that  the  amount  of  work  they  have  interferes  with  how  well  the  work  is  done)  that 
employees  reported  a  higher  level  of  Job  Factors  than  are  actually  present  in  the  work. 

The  Organizational  Rating  is  based  on  responses  to  this  section  and  is  not  used  in  the  Survey 
Priority  Rank  calculation.  The  organizational  ratings  are  not  directly  incorporated  into  the 
Survey  Priority  Rank  calculation  in  order  to  retain  a  job  factor  to  discomfort  factor  link  in 
determining  the  ranks.  Maintaining  these  ratings  as  a  separate  item  allows  EWG  to  make  the 
decision  regarding  EPRA  status  based  on  the  likelihood  that  organizational  factors  are 
contributing  to  discomfort  without  the  presence  of  intervening  risk  factors. 

2.2.2.3  Physical  Effort 

The  following  paragraphs  describe  the  research  findings  associated  with  developing  this  portion 
of  the  Survey,  as  well  as  the  purpose  of  the  questions,  the  rationale  for  the  responses,  and  what 
the  results  indicate.  The  Physical  Effort  section  is  included  as  Figure  2.4. 
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Figure  2.4  Physical  Effort 


C.  PHYSICAL  EFFORT 


45 .  How  would  you  describe  the  physical  effort  required  of  your  job? 
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2.2.2.3.1  Research  Findings 


The  scale  used  in  the  Survey  is  a  version  of  the  Borg  Scale  (Borg,  1970  [20]).  The  specific  terms 
were  adopted  based  on  a  modified  Borg  scale,  which  has  been  shown  to  be  effective  in  obtaining 
ratings  of  perceived  exertion.  Because  this  scale  was  taken  directly  from  a  validated,  widely 
used  source,  detailed  re-validation  of  this  question  was  not  performed. 

2.2.2.3.2  Question  (Q45) 

Question  45  asks  the  employee  to  classify  the  overall  physical  effort  required  by  tasks  that  are 
performed  an  approximately  daily  basis.  This  question  provides  a  good  indication  of  employees’ 
perception  of  work  load/work  demands  throughout  the  shop. 

2.2.2.3.3  Responses 

There  are  15  response  choices,  from  6  -  no  exertion  at  all,  to  20  -  maximal  exertion. 

2.2.2.3.4  What  the  Section  Indicates 

The  shop  result  from  this  section  provides  an  indication  of  how  “easy”  or  “hard”  the  employees 
think  the  work  is.  The  Air  Force  may  use  this  information  in  post  hoc  analyses  to  determine  how 
employee  perception  of  work  effort  corresponds  to  reported  injuries  or  illnesses.  The  Physical 
Effort  Score  for  the  shop,  based  on  responses  to  question  45  is  not  used  in  the  Priority  Rank 
calculation.  The  physical  effort  is  not  used  directly  in  the  Priority  Rank  calculations  because  the 
amoirnt  of  physical  effort  can  be  misleading  relative  to  WMDs.  A  highly  repetitive  hand  and 
finger  task  may  require  little  overall  physical  effort  because  the  forces  exerted  are  low  and  the 
body  mass  moved  is  small,  while  contributing  to  WMDs.  Alternatively,  a  task  may  require  a 
large  variety  of  moderately  forceful  whole  body  movements.  Because  of  the  task  variety  and  the 
nature  of  the  tasks,  the  risk  for  WMD  may  be  minimal  while  the  effort  required  is  high. 

2.2.3  Part  II:  Your  Body’s  Response  to  Work  Demands 

This  section  enables  the  employee  to  identify  the  occurrence,  location,  frequency,  and/or  degree 
of  discomfort  that  may  be  associated  vrith  daily  work  activities. 
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Part  II  is  divided  into  two  sections:  Discomfort  Factors  and  General  Questions. 

2.2.3.1  Discomfort  Factors 

The  following  paragraphs  describe  the  research  findings  associated  with  developing  this  portion 
of  the  Survey,  as  well  as  the  purpose  of  the  questions,  the  rationale  for  the  responses,  and  what 
the  results  indicate.  The  Discomfort  Factors  section  is  included  as  Figure  2.5. 

2.2.3.1.1  Research  Findings  The  Discomfort  Factors  section  was  based  on  lessons  learned 
from  available  symptom  survey  tools  and  the  literature.  The  final  form  of  the  section  was  also 
shaped  by  feedback  firom  survey  test  participants  and  the  Air  Force. 

The  symptom  survey  included  in  the  (draft)  ANSI  Z-365  Standard  for  the  Control  of  Cumulative 
Trauma  Disorders  [17]  provided  a  layout  and  format  concept  for  the  Discomfort  Factors  section. 
This  format  was  selected  because  it  is  logical  to  ask  questions  by  body  zone.  The  format,  which 
was  improved  by  adding  illustrations,  allows  the  survey  participant  to  quickly  identify  the  body 
zone  presented.  The  concept  of  shading  the  intended  body  zone  within  a  whole  body  illustration 
was  derived  from  Kuorinka  et  al,  1987  [21]. 

One  of  the  major  decisions  for  the  Discomfort  Factors  section  was  to  include  questions  on  both 
the  severity  and  frequency  of  a  particular  physical  complaint.  Many  of  the  surveys  found  in  the 
literature  considered  only  severity  or  frequency,  not  both.  Both  the  ANSI  Z-365  [17]  and  the 
Johnson  and  Johnson  (Johnson  and  Johnson,  1995  [22])  surveys  address  both  frequency  and 
severity  issues. 
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Other  questions  included  in  many  of  the  surveys  found  in  the  literature  were  either  variations  of  a 
frequency  or  severity  question  or  questions  which  would  not  provide  information  useful  to  the 
Air  Force.  Therefore  these  questions  were  not  further  considered  for  incorporation  into  the 
Survey. 

The  severity  scale,  “Mild,  Moderate,  Severe,”  was  derived  from  the  Johnson  and  Johnson 
symptom  survey.  This  scale  was  used  because  it  is  a  three-point  scale  and  the  terminology  is 
easy  to  imderstand. 

The  frequency  scale,  “Daily,  Weekly,  Monthly,”  was  derived  from  Marley  and  Kumar  (1996 
[23]).  Marley  and  Kumar  used  “Constantly  (nearly  every  day),  Frequently  (a  few  times/week), 
and  Rarely  (a  few  times/month).”  Again,  the  three-point  scale  was  selected  for  its  simplicity. 
The  terminology  was  simplified  to  make  it  easier  to  understand. 

The  ANSI  Z.365  symptom  survey  provided  the  basis  for  Q46-Q60.  In  addition,  the  work  of 
Dickinson  et  al  (1992  [24])  provided  helpful  guidance  in  the  development  of  these  questions. 
These  questions  were  also  modified  to  reflect  feedback  obtained  during  Survey  testing.  One 
change  made  in  response  to  feedback  was  to  eliminate  the  need  for  the  employee  to  answer 
questions  that  were  not  relevant  to  his/her  experience.  To  enable  the  user  to  complete  the  section 
as  quickly  as  possible,  a  question  was  added  (Questions  46,  49,  52,  55,  and  58)  prior  to  the 
frequency  and  severity  questions  for  each  body  zone.  The  question  asked  if  the  employee  had 
experienced  any  discomfort,  pain,  etc.  If  the  answer  was  no,  the  employee  is  directed  to  skip  the 
“frequency”  and  “severity”  questions  for  that  particular  body  zone  since  those  questions  would 
not  be  relevant.  Another  significant  change  was  to  divide  the  questions  into  two  sections: 
Discomfort  Factors  and  General  Questions.  Discomfort  Factors  were  limited  only  to  the 
frequency  and/or  severity  of  pain,  discomfort,  etc.  The  separation  was  made  to  increase  the 
speed  of  the  scoring  process:  the  Discomfort  Factors  are  used  in  determining  the  Discomfort 
Rating  while  the  General  Questions  may  be  used  to  interpret  the  Discomfort  Rating. 

2.2.3.1.2  Questions  (Q46-Q60)  In  Section  D,  the  employee  is  asked  to  respond  to  questions 
that  relate  to  the  occurrence,  frequency,  and  severity  of  discomfort,  fatigue,  numbness,  or  pain  in 
each  of  the  five  body  zones.  The  introduction  at  the  top  of  page  7,  “This  section  enables  you  to 
identify  how  your  body  responds  to  the  demands  of  your  job,”  is  included  to  focus  the 
employee’s  responses  to  job-related  occurrence  of  any  symptoms.  (In  section  E  of  the  Survey, 
information  is  obtained  that  may  help  identify  other  potential  sources  of  symptoms.) 

The  employee  symptoms  survey  included  in  the  proposed  ANSI  Z.365  standard  [17]  was  used  as 
the  basis  for  the  design  of  the  Discomfort  Factors  section.  A  picture  of  the  relevant  body  zone  is 
provided  along  with  the  questions  to  ensure  that  the  employee  is  able  to  pinpoint  the  location  of 
any  discomfort,  pain,  etc.,  on  his/her  body,  then  communicate  that  information  in  a  consistent 
way  using  questions  46-60.  Body  zones  were  used  instead  of  body  parts  (e.g.,  hand,  thigh,  etc.) 
or  joints  (e.g.,  knee,  elbow)  to  eliminate  the  requirement  of  having  employees  determine  the 
precise  location  of  their  symptoms  The  concept  of  using  shading  to  help  employees  identify  the 
body  zones  was  adopted  based  on  work  of  Kourinka  et  al  (1987)  [21].  Both  concepts  were  used 
to  minimize  employee  response  time. 
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2.2.3.1.3  Responses  The  employee  is  directed  to  answer  questions  “by  body  zone.”  A 
response  of  “no”  to  the  first  question  directs  the  employee  to  proceed  to  the  next  column.  If  the 
employee  responds  “yes”  to  the  first  question,  he/she  is  then  asked  to  describe  the  “frequency” 
and  “severity”  of  the  symptoms. 

The  Criteria  Table  used  to  score  the  Discomfort  Factors  section  is  based  on  the  work  of  Marley 
and  Kumar  [23].  These  researchers  conducted  tests  to  determine  the  relationship  between  a 
person’s  score  on  a  discomfort  assessment  tool  and  his/her  likelihood  to  seek  treatment.  The 
research  produced  a  Criteria  Table  based  on  discriminant  analysis  which  categorizes  the 
likelihood  of  seeking  treatment  based  on  severity  and  frequency  scores.  The  current  Criteria 
Table  was  developed  by  modifying  the  severity  scale  to  a  three-point  scale.  Further  adjustment 
of  the  Criteria  Table  was  also  made  to  increase  specificity  (e.g.,  make  it  more  likely  to  identify 
personnel  likely  to  seek  treatment). 

The  Discomfort  (Factors)  Rating  scale.  Low:  <  or  =  30%,  Medium:  31-60%,  and  High:  >  or  = 
61%,  is  also  based  on  the  analysis  conducted  by  Marley  and  Kumar  [23].  The  analysis  indicated 
that,  of  the  people  who  sought  medical  treatment,  64.7%  scored  in  the  “very  likely  to  seek 
treatmenf’  category.  Of  the  people  who  did  not  seek  medical  treatment,  33.4%  scored  in  the 
“very  likely  to  seek  treatment”  category.  This  data  was  adapted  to  identify  the  Discomfort 
Rating  decision  points. 

2.2.3.1.4  What  the  Section  Indicates  Results  from  this  section  indicate  the  common 
experience  with  discomfort  and/or  other  physical  symptoms  for  employees  as  a  group.  Since  the 
data  is  interpreted  for  the  group  as  a  whole,  there  should  be  little  cause  for  concern  about  the 
validity  or  influence  of  individual  employee  reports. 

Just  as  was  done  for  the  Job  Factors  section,  grouping  the  Discomfort  Factors  by  body  zones  also 
helps  identify  the  body  zone(s)  in  which  employees  may  be  experiencing  possible  symptoms  of 
WMDs  to  the  greatest  extent.  Again,  this  will  help  Public  Health  and  the  EWG  establish  targets 
for  effective  problem  solving  strategies  for  shops  which  are  upgraded  to  EPRA  status 

The  Discomfort  Rating  that  is  determined  for  each  body  zone  is  used  together  with  the  Risk 
Factor  Rating  to  determine  the  overall  Priority  Rank  for  the  shop. 

2.2.3.2  General  Questions 

The  following  paragraphs  describe  the  purpose  of  the  questions,  the  rationale  for  the  responses, 
and  what  the  results  obtained  from  this  portion  of  the  Survey  indicate.  The  General  Questions 
section  is  included  as  Figure  2.6. 
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Figure  2.6  General  Questions 


E.  GENERAL  QUESTIONS 


61 .  In  the  past  12  months,  have  you  seen  a  health  care  provider  for  any  pain  or  discomfort  that  you  think  relates  to  your  job?  Yes  O  No  O 

62.  Do  you  experience  any  work-related  pain  or  discomfort  that  docs  not  improve  when  you  are  away  from  work  ovcmite  or  over  the 

weekend?  Yes  O  No  O 

63.  In  the  past  12  months,  has  any  work-related  pain  or  discomfort  caused  you  difficulty  in  carrying  out  normal  activities  (e.g.,  job, 

hobby,  leisure,  etc.)?  Yes  O  No  O 

64.  Has  a  health  care  provider  ever  told  you  that  you  have  any  of  the  following  conditions  which  you  think  might  be  related  to  your 

work?  Yes  O  No  O 

•  Tendonitis/Tenosynovitis  •  Ganglion  Cyst  •  Trigger  Finger  •  •  Overuse  Syndrome 

•  Epicondylitis  (Tennis  Elbow)  •  Bursitis  •  Carpal  Tunnel  Syndrome 

•  TTioracic  Outlet  Syndrome  •  Back  Strain  •  Knee  or  Ankle  Strain 

65.  Do  you  have  or  have  you  ever  had  one  or  more  ofthe  following  conditions?  Yes  O  No  O 

•  Wrist  Fracture  •  Rheumatoid  Arthritis  •  Diabetes 

•  Thyroid  Disorder  •  Hypertension  •  Kidney  Disorders 

•  Gout 


2.2.3.2.1  Questions  (Q61-65)  The  questions  in  this  section  were  developed  as  a  result  of 
several  discussions  with  the  Air  Force.  The  purpose  of  the  questions  is  to  enable  employees  to 
provide  background  information  (e.g.,  impact  of  non-work-related  activities,  other  physical 
conditions  that  may  be  the  source  of  the  discomfort,  etc.)  on  reported  discomfort,  fatigue,  etc. 
Again,  since  only  the  results  from  the  shop  as  a  whole  will  be  interpreted,  the  questions  are  not  to 
be  used  to  make  conclusions  about  an  individual  employee  or  case. 

2.2.3.2.2  Responses  The  format  of  the  questions  was  chosen  to  enable  employees  to  respond 
with  a  “yes”  or  “no”  answer. 

2.2.3.2.3  What  the  Section  Indicates  Responses  to  these  questions  are  not  factored  into  the 
Priority  Rank  calculation.  They  are  important,  however,  in  that  they  offer  a  means  for  Public 
Health  to  interpret  potentially  wide  discrepancies  between  the  Risk  Factor  Rating  and  the 
Discomfort  Rating  for  a  shop.  For  example, 

•  the  high  Priority  Rating  for  the  shop  (based  on  high  Risk  Factor  and  Discomfort  Ratings) 
and  the  lack  of  illness/injury  cases  reported  for  the  shop  may  indicate  that  employees  are 
under-reporting  symptoms  of  WMD; 

•  pre-existing  conditions  or  injuries  (whether  they  are  work-related  or  otherwise)  may  be 
responsible  for  a  high  Discomfort  Rating  for  the  shop;  and 

•  work-related  discomfort  or  pain  has  made  it  difficult  for  employees  who  work  in  this 
shop  to  carry  out  their  normal  activities. 
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2.2.4  Part  HI:  Work  Content 


TTiis  section  enables  employees  to  provide  a  basic  description  of  work  performed  in  the  shop 

The  following  paragraphs  describe  the  purpose  of  the  questions,  the  rationale  for  the  responses, 
and  what  the  results  obtained  from  this  portion  of  the  survey  indicate.  An  excerpt  from  the  Work 
Content  section  is  included  as  Figure  2.7. 


Figure  2.7  Work  Content 

The  section  below  will  enable  you  to  describe  the  content  of  the  work  that  you  do  in  your  current  shop. 
Fill  in  the  box  that  describes  how  frequently  you  do  the  task  listed,  based  on  the  following  definitions: 

•  Routine:  Performed  on  three  or  more  days  per  week. 

•  Non-routine:  Performed  two  days  a  week  or  less. 

•  Seasonal:  Performed  only  during  certain  times  of  the  year. 

•  Never/NA:  You  do  not  perform  this  type  of  work. 


Tvpe  of  Work 

Work  Frequencv 

(Check  one) 

Routine 

Non-Routine 

Seasonal 

Never/NA 

abrading 

O 

O 

O 

O 

baking 

O 

o 

O 

O 

bolting/screwing 

O 

o 

O 

o 

calling  (telephone  use) 

O 

o 

O 

o 

chipping 

O 

o 

O 

o 

2.2.4.1  Questions 

The  types  of  work  listed  (Q66-Q120)  were  taken  from  the  Revised  Handbook  for  Analyzing 
Jobs.  In  some  cases,  the  task/title  has  been  modified  for  clarity  (e.g.,  for  Q69  -  calling, 
“telephone  use”  was  added  to  the  task  title)  or  grouped  according  to  similar  expected  risk  factor 
exposure  (e.g.,  for  Q86,  the  task  titles  for  grinding,  buffing,  and  polishing  were  combined). 
Blank  lines  for  Q121  and  Q122  are  provided  to  enable  employees  to  “write  in”  task  types  that 
were  not  included  on  the  list. 

2.2.4.2  Responses 

Four  response  choices  are  provided:  routine,  non-routine,  seasonal,  and  never/NA.  The 
following  definitions  were  adopted: 
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•  Routine  tasks:  perfomed  three  or  more  days  per  week. 

•  Non-routine  tasks:  performed  two  days  a  week  or  less. 

•  Seasonal  tasks:  performed  only  during  certain  times  of  the  year. 

Routine  was  originally  defined  as  “performed  on  an  approximately  daily  basis”  by  OSHA  [16]. 
The  Air  Force  removed  the  need  for  the  user  to  interpret  “approximately  daily”  by  suggesting  the 
final  definition,  “performed  three  or  more  times  per  week.”  The  definitions  for  non-routine  and 
seasonal  were  also  provided  by  the  Air  Force  to  reflect  the  wide  variety  of  work  duration’s  found 
at  Air  Force  installations.  During  initial  testing.  Survey  participants  expressed  the  need  for  a 
response  choice  to  each  task  type.  The  never/NA  response  choice  was  added  based  upon  this 
participant  input.  This  was  a  way  of  ensuring  that  the  employee  considered  every  task  on  the  list 
when  communicating  work  content. 

2.2.4.3  What  the  Section  Indicates 

The  Section  serves  two  purposes.  The  first  purpose,  as  stated  above,  is  to  provide  Public  Health 
and  the  EWG  some  direction  on  which  tasks  could  become  the  focus  of  further  investigation  or 
problem-solving  for  EPRA-designated  shops.  Employees  may  perform  10-15  different  tasks  on 
a  routine  basis.  Responses  in  this  section,  however,  provide  no  insight  into  whether  or  not 
employees  believe  that  the  tasks  may  be  the  source  of  exposure  to  ergonomic  hazards.  The 
second  purpose  is  to  prepare  the  employees  to  respond  to  Part  IV.  In  short.  Part  III  describes 
which  tasks  the  employees  do.  Part  IV  describes  which  tasks  the  employees  think  may  be  good 
targets  for  improvement. 

2.2.5  Part  IV:  Process  Improvement  Opportunities 

This  section  enables  employees  to  describe  the  activities  that  he/she  believes  place  the  greatest 
demands  on  the  body.  Responses  may  be  used  by  Public  Health  and  the  EWG  to  prioritize 
problem-solving  activities  within  a  shop. 

The  following  paragraphs  describe  the  purpose  of  the  questions,  the  rationale  for  the  responses, 
and  what  the  results  obtained  from  this  portion  of  the  Survey  indicate. 


2.2.5.1  Questions  fl-4^ 

Four  questions  are  presented  to  elicit  the  maximum  number  of  responses  from  employees  with  a 
variety  of  attitudes  toward  work.  For  example,  some  employees  may  believe  that  it  is  a  poor 
reflection  on  their  own  physical  condition  to  admit  that  their  work  or  part  of  their  work  makes 
them  sore.  Other  employees  wait  for  any  opportunity  to  complain  about  every  ache  or  pain  they 
experience,  whether  it  is  work-related  or  not.  Since  these  points  of  view  exist  and  since  the 
purpose  of  the  section  is  to  provide  Public  Health  and  the  EWG  with  information  on 
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improvement  opportunities,  the  questions  are  worded  to  elicit  the  maximum  number  of 
responses. 

2.2.5.2  Responses 

The  employee  is  asked  to  write  the  response  in  the  space  provided.  Previous  trials  of  the  Survey 
indicate  that  many  employees  will  offer  constructive  suggestions  for  addressing  “problem”  jobs 
or  tasks  in  the  work  area.  In  most  cases,  employees  will  not  write  a  response  to  all  of  the 
questions. 

2.2.5.3  What  the  Section  Indicates 

A  large  number  of  responses  to  the  questions  in  this  section  makes  the  task  of  establishing 
problem-solving  priorities  much  easier.  For  example.  Public  Health  may: 

•  observe  (or  obtain  a  description  of)  the  task; 

•  review  the  Survey  results  to  see  if  the  demands  of  the  task  are  consistent  with  the  Job 
Factor  and  Discomfort  Ratings  reported  for  the  shop; 

•  if  the  task  demands  and  Ratings  are  consistent,  the  task  may  be  a  source  for  employee 
exposure  to  ergonomic  risk  factors;  and/or 

•  recommend  that  the  task  be  selected  for  further  investigation  or  immediate  problem¬ 
solving. 

Public  Health  summarizes  the  task  list  as  part  of  the  report  to  communicate  shop  needs  to 
Bioenvironmental  Engineering  Services. 

2.3  Risk  Rating  and  Prioritization 

The  following  sections  describe  the  development  of  the  Survey  scoring  procedures,  including  the 
research  findings  and  scoring  process  design.  Individual  sections  are  devoted  towards  describing 
the  scoring  procedures  for  each  section  of  the  Survey. 


2.3.1  Research  Findings  and  Scoring  Process  Overview 

As  previously  stated,  validated  methods  for  obtaining  risk  ratings  and  priorities  from  risk  factor 
survey  tools  are  lacking  in  the  scientific  literature.  Some  information  is  available  regarding  the 
likelihood  of  seeking  medical  treatment  based  on  the  frequency  and  severity  of  discomfort  [23]. 

The  rationale  for  determining  concerns  related  to  discomfort  and  scoring  for  the  Survey  is  baised 
on  the  methods  used  by  Marley  and  Kumar  [23].  While  concerns  are  identified  on  an  individual 
basis  and  indicated  by  single  tally  marks  on  the  scoring  forms,  the  risk  statement  is  based  on  the 
percent  of  people  indicating  concerns.  Several  key  concerns  in  the  selection  and  development  of 
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all  scoring  methods  were  the  speed  and  ease  of  use.  This  method  of  determining  a  risk  statement 
or  “rating”  satisfies  the  Air  Force’s  desire  to  place  priority  attention  on  work  areas/shops  where 
the  greatest  proportion  of  employees  are  reporting  discomfort. 

Scoring  for  the  Job  Factors  section  was  designed  to  be  as  similar  as  possible  to  the  scoring  of  the 
Discomfort  Factors  section.  Consistency  between  sections  maximizes  the  speed  and  ease  of 
scoring.  Threshold  levels  were  set  for  each  body  area  based  on  the  consensus  judgment  of  a 
team  of  experienced  ergonomists.  The  threshold  levels  establish  the  number  of  risk  factors  (to 
which  the  employee  is  exposed)  above  which  an  ergonomist  would  consider  performing  an 
additional  evaluation. 

For  example,  if  an  employee  recorded  that  his/her  job  involved  three  of  the  job  factors,  each  for 
over  two  hours  a  day,  the  ergonomists  concluded  that  the  potential  for  reports  of  shoulder 
discomfort  may  increase  --  enough  to  warrant  additional  investigation.  The  consensus  judgment 
decisions  are  based  on  approximately  45  years  of  combined  experience  in  industrial  and 
administrative  area  ergonomic  field  work.  Threshold  levels  are  contained  in  Table  2.2. 

Table  2.2  Job  Factor  Threshold  Levels  By  Body  Zone. 


Body  Zone 

Threshold  Level 

Shoulder/Neck 

>2 

Hand/Wrist/Arm 

>4 

Back/Torso 

>2 

Legs/Feet 

>  1 

Head/Eyes 

>  1 

For  simplicity  of  scoring,  each  of  the  Risk  Factor  (Job  Factor)  questions  have  equal  “weight.” 
Initial  usability  testing  indicated  that  the  increase  in  scoring  complexity  due  to  “weighting”  the 
job  factor  question  (e.g.,  high  force  is  “worse  than  ”  stressful  positions  did  not  increase  the  value 
of  the  output  to  a  commensurate  level).  As  is  the  case  for  the  Discomfort  Factors  section,  these 
individual  “concerns”  are  converted  to  shop  percentages  to  determine  risk  statements. 

The  overall  Survey  Priority  Rank  for  a  shop  is  based  on  the  combination  of  the  percentage  of 
people  experiencing  discomfort  (at  a  level  of  concern)  and  the  percentage  of  people  exposed  to 
risk  factors  (at  a  level  of  concern).  The  highest  priorities  are  given  to  those  shops  where  the  body 
areas  of  discomfort  present  correspond  to  the  body  areas  of  ergonomic  risk  factors  exposure. 
The  Discomfort  Ratings  are  weighted  more  heavily  than  the  Risk  Factor  Ratings  when 
determining  priority  for  further  analysis  or  intervention,  as  is  consistent  with  ANSI  Z-365  [17]. 
This  is  reflected  in  the  design  of  the  Priority  Matrix  table  which  is  used  to  determine  the  overall 
Survey  Priority  Rank. 

A  number  of  other  factors  are  presented  to  help  interpret  the  Survey  Priority  Rank  and  design  the 
appropriate  strategy  for  follow-up.  The  final  determination  of  EPRA  status  is  based  on  the 
judgment  of  the  EWG  rather  than  directly  from  a  score  obtained  by  the  Survey.  The  EWG  can 
evaluate  the  Survey  Priority  Ranks,  the  other  considerations,  and  their  knowledge  of  shop 
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activities  to  make  the  final  determination.  The  importance  of  integrating  multiple  data  sources  to 
make  a  decision  rather  than  relying  upon  a  single  score  has  been  stressed  by  several  researchers 
(Drury,  1990  [25];  Kirwan  &  Ainsworth,  1992  [26]). 

2.3.2  Scoring  Process  Design 

The  scoring  process  design  resulted  from  discussions  between  the  designers  and  the  Air  Force. 
Several  iterations  were  necessary  since  validated  methods  for  obtaining  “risk”  scores  fi'om  the 
use  of  survey  tools  is  lacking  in  the  scientific  literature.  Design  decisions  were  made  to 
maximize  ease  and  speed  of  scoring  and  to  provide  Public  Health  with  a  standardized  method  for 
comparing  and  prioritizing  the  potential  risk  of  WMD  development  for  all  PEPA  shops 
throughout  an  installation. 

The  scoring  process  is  performed  for  each  part  of  the  Survey  separately.  Results  are  then  used  to 
determine  an  overall  Survey  Priority  Rank  for  the  shop  and  to  help  provide  a  recommendation 
for  the  EWG.  The  scoring  process  will  be  discussed  in  the  following  sections: 

•  Parti:  Description  of  Work; 

•  Part  II:  Your  Body’s  Response  to  Work  Demands; 

•  Part  III:  Work  Content;  and 

•  Part  IV :  Process  Improvement  Opportunities. 


2.3.2.1  Parti;  Description  of  Work 

Part  I  is  divided  into  three  sections:  Risk  Factor  Rating,  Organizational  Factor  Rating,  and 
Physical  Effort  Score. 

2.3.2.1.1  Risk  Factor  Rating  The  scoring  sheet  used  to  calculate  the  Risk  Factor  Rating  is 
shown  in  Figure  2.8. 

2.3.2.1.1.1  Rationale  Criteria  levels  were  established  for  each  body  zone  based  on  a  consensus 
judgment  of  a  team  of  experienced  ergonomists  from  The  Joyce  Institute/A  Unit  of  Arthm  D. 
Little,  Inc.  The  criteria  levels  establish  the  number  of  risk  factors  (to  which  the  employee  is 
exposed)  above  which  an  ergonomist  would  consider  performing  an  additional  evaluation.  For 
simplicity  of  scoring,  each  of  the  risk  factor  (job  factor)  questions  have  equal  “weight.” 

2.3.2.1.1.2  Process  A  scoring  sheet  is  provided  for  the  technician  to  record  tallies  for  each 
body  zone  based  on  individual  employee  responses.  For  example,  the  technician  reviews  the 
shoulder/neck  job  factor  responses  from  one  employee.  The  technician  coimts  the  number  of 
questions  that  the  employee  responded  2-4  hours  or  4-8  hours.  (Check  marks  placed  in  the  “0-2 
hours”  or  “Never/NA”  columns  are  not  cormted  as  is  consistent  with  the  OSHA  draft  checklist.) 
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If  the  number  of  check  marks  is  greater  than  2,  the  technician  makes  one  tally  mark  in  the 
shoulder/neck  tally  box  on  the  scoring  sheet.  The  technician  continues  the  process  by  reviewing 
the  responses  made  by  every  shop  employee  for  every  other  body  zone.  Tallies  are  converted 
into  a  Low,  Medium,  or  High  Risk  Factor  Rating. 

Each  body  zone  Risk  Factor  Rating  is  used  to  determine  the  Survey  Priority  Rank  for  the  shop. 

2.3.2.1.2  Organizational  Factor  Rating  The  scoring  sheet  used  to  calculate  the 
Organizational  Factor  Rating  is  shown  in  Figure  2.9. 
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Figure  2.8  Scoring  Sheet  -  Job  Factors 


Part  I  -  Job  Factors 

A  -  Risk  Factor  Ratings  (Questions  1  -  38) 

HHHHHHHIii 

Step  I  Step  2 

Step  3 

Step  4 

For  each  body  area,  count  the  number  of 
responses  in  the  2-4  hour  column  and  in  the 
4-8  hour  column.  If  that  number  exceeds 
the  criteria  number  in  the  box  in  the  upper 
r^t,  make  one  tally  marie.  Place  only  one 
mark  per  survey  in  each  box.  Write  the  total 
of  tile  tallies  in  the  Total  box. 


Divide  the  Total  tallies  by 
the  number  of  surveys 
from  one  shop. 


Multiply  that  number 
by  100  to  get  the 
percentage. 


Write  the  Risk  Factor 
Rating  (Low,  Med,  High) 
in  the  box  for  each  body 
part  using  the  scale  below. 


Low 


jm 


mgk 

<30%  31-60% 
61+%  -  ' 


Shoulder/Neck  Tally  Box 
Questions  1-7 


number  of 
surveys 


Total 


xl00  = 


% 


A.1  Shoulder/Neck 
Risk  Factor  Rating 


HandAVrist/Arm  Tally  Box 

Questions  8-21 


number  of 
surveys 


Total 


xl00  = 


% 


Back/Torso  Tally  Box 
Questions  22-30 


number  of 
surveys 


Total 


xl00  = 


% 


A.  2  Hand/Wrist/Arm 
Risk  Factor  Rating 


A.3  Back/Torso 
Risk  Factor  Rating 


Legs/Feet  Tally  Box 
Questions  3 1-34 


number  of 
surveys 


Total 


xl00  = 


% 


Head/Eyes  Tally  Box 
Questions  35-38 


number  of 
surveys 


Total 


xl00  = 


% 


A.4  Legs/Feet 
Risk  Factor  Rating 


A.5  Head/Eyes 
Risk  Factor  Rating 
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Figure  2.9  Scoring  Sheet  -  Organizational  Factors 


Part  I 

B  -  Organizational  Factor  (Questions  39-44) 


Step  I  Step  2  Step  3  Step  4 


Step  5 


For  each  question  that  has  a 
response  of  a  4-Agree  or  5- 
Strongly  Agree,  make  a 
tally  in  the  tally  box.  Write 
the  total  tallies  in  the  Total 
box. 

Tally  Box  1 


Divide  by  6  Divide  by 

number 
surveys  from 
shop. 


the  Multiply  that  number 
of  by  100  to  get  the 
one  percentage. 


Write  the  Organizational 
Factor  Rating  (Low,  Med, 
High)  in  the  box  based  on 
the  scale  below: 


JHigh 

^0% 

31-60% 

61+%' 

B.  Organizational  Factor 
Rating 


2.3.2.1.2.1  Rationale  Responses  of  “Agree”  or  “Strongly  Agree”  indicate  the  presence  of  an 
organizational  factor  (e.g.,  “I  often  feel  that  I  ■will  not  be  able  to  satisfy  the  conflicting  demands 
of  various  people  around  me”)  that  may  influence  the  employee’s  responses  to  the  Job  Factor, 
Discomfort,  or  other  questions.  The  greater  the  number  of  responses,  the  higher  the  potential  that 
organizational  factors  (or  “stress”)  are  impacting  employees.  Again,  it  should  be  noted  that  the 
scoring  of  the  Organizational  Factors  section  is  based  on  responses  from  the  entire  group,  rather 
than  from  a  single  employee. 

2.3.2.1.2.2  Process  A  scoring  sheet  is  provided  for  the  technician  to  tally  the  Organizational 
Factors.  For  each  question  that  has  a  response  of  “Agree”  or  “Strongly  Agree,”  the  technician 
makes  a  tally  in  the  tally  box.  After  the  process  has  been  completed  for  each  question/for  each 
employee,  a  total  tally  is  calculated.  This  number  is  used  to  determine  the  Organizational  Factor 
Rating  of  Low,  Medium,  or  High. 

The  Organizational  Factor  Rating  is  not  used  in  calculating  the  Survey  Priority  Rank  for  the 
shop. 

The  Organizational  Factor  Rating,  however,  can  be  used  in  interpreting  the  Survey  Priority  Rank. 
For  example,  if  the  shop  Survey  Priority  Rank  is  a  6  because  of  a  Low  Risk  Factor  Rating  and  a 
High  Discomfort  Rating,  and  the  Organizational  Factor  Rating  is  also  High,  this  may  indicate 
that  the  high  presence  of  Organizational  Factors  may  be  causing  employees  to  report  a  higher 
level  of  discomfort  than  the  relatively  low  presence  of  Risk  Factors  indicates.  (Note:  this  is  only 
one  interpretation.  Another  reason  for  a  High  Discomfort  Rating,  Low  Risk  Factor  Rating,  and  a 
High  Organizational  Factor  Rating  is  that  many  of  the  employees  from  a  shop  could  have  had 
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past  illnesses/injuries  whose  s5anptoms  have  not  disappeared  even  though  the  jobs  have  been 
improved  to  minimize  the  presence  of  risk  factors.) 

2.3.2.1.3  Physical  Effort  Score  The  scoring  sheet  used  to  calculate  the  Physical  Effort  Score 
is  shown  in  Figure  2.10. 

2.3.2. 1.3.1  Rationale  The  actual  numerical  response  is  used  to  generate  the  Physical  Effort 
Score.  The  higher  the  number/score  averaged  across  the  entire  shop  is,  the  higher  the  amount  of 
physical  effort  that  employees  feel  is  required  in  that  shop. 

2.3.2.1.3.2  Process  A  scoring  sheet  is  provided  for  the  technician  to  tally  the  Physical  Effort 
Score.  The  technician  writes  the  actual  numeric  response  from  each  Survey  into  the  tally  box  on 
the  scoring  sheet.  The  Physical  Effort  Score  is  calculated  by  totaling  the  responses  and 
determining  the  average  response  for  the  shop. 

The  Physical  Effort  Score  is  not  used  in  calculating  the  Survey  Priority  Rank  for  the  shop. 

The  Physical  Effort  Score  may  be  used  by  the  Air  Force  in  post  hoc  analysis  to  determine  a 
relationship  between  employee  reported  discomfort  and  perceived  exertion. 

Figure  2.10  Scoring  Sheet  -  Physical  Effort  Score 


Part  I 

C  -  Physical  Effort  (Question  45) 


_ _  I 


Step  1 

Step  2 

Step  3 

Write  the  numeric  score  (6-20)  for 
each  survey  in  the  tally  box.  Add  the 
numbers  and  write  the  total  in  the  total 
box. 

Divide  that  total  by  the 
number  of  surveys. 

Write  the  average  in  the  Physical  Effort 
box. 

Tally  Box 

number 

C. 

Physical  Effort  Score 

Total 

of  surveys 
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2.32.2  Part  II:  Your  Body’s  Response  to  Work  Demands 


Part  II  is  divided  into  two  sections:  Discomfort  Rating  and  General  Questions  Scores. 


2.3.2.2.1  Discomfort  Rating  The  scoring  sheet  used  to  calculate  the  Discomfort  Rating  is  shown  in  Figure 

2.11. 


2.3.2.2.1.1  Rationale  A  Criteria  Table  is  used  to  determine  a  tally,  by  body  zone,  for  each  Survey  from  the 
shop.  The  Criteria  Table  was  designed  based  on  the  work  of  Marley  and  Kiunar  [23]  and  is  discussed  in  Section 
3  on  pp.  3-13.  Use  of  the  Criteria  Table  enables  the  technician  to  “coimt”  reports  of  discomfort,  pain,  etc.  that 
are  experienced  on  a  daily  basis,  that  are  moderate  or  severe  and  experienced  on  a  weekly  basis,  or  that  are 
severe  and  experienced  on  a  monthly  basis.  Employees  whose  responses  to  Q46-Q60  fall  into  any  of  these 
categories  are  more  likely  to  seek  medical  treatment  than  those  who  do  not  report  lower  frequency/discomfort 
experience. 
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Figure  2.11  Scoring  Sheet  -  Discomfort  Rating 


Part  II  -  The  Body's  Response 
D  -  Discomfort  Rating  (Questions  46 ' 


60) 


Step  I 


For  each  body  part,  look  at  the  responses  to  the 
second  and  third  questions  (47  Sc  48,  50&51, 
53&54,  56&57,  59&60),  If  participants  have 
answered  them,  then  look  at  the  Criteria  Table.  If 
the  combination  of  answers  fits  one  of  the 
categories,  then  make  a  tally  mark  in  the  tally  box 
for  each  body  part.  For  example:  if  47  is  "weekly" 
and  48  is  "moderate"  then  make  a  tally  mark. 
Count  and  put  total  in  Total  box. 


Daily 


Weekly 


Monthly 


Step  2 


Divide  the  total  tallies 
by  the  number  of 
surveys  from  one  shop. 


Step  3 


Multiply  that  number  by 
100  to  get  the  percentage. 


Step  4 


Write  the  Discomfort  Rating 

(Low, 

Med,  High)  in  the  box  for  each  body  I 

part  using  the  scale  below. 

Low  Med  High 

<30%  31-60%  61+% 

Criteria  Table 


Shoulder/Neck  Tally  Box 
Question  46-48 

Total 

HandAVrist  Arm  Tally  Bo 
Question  49-51 

X 

Total 

Back/Torso  Tally  Box 
Question  52-54 

Total 

Legs/Feet  Tally  Box 
Question  55-57 

Total 

Head/Eyes  Tally  Box 
Question  58-60 

Total 

number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


number  of 
surveys 


X  100  = 


xl00  = 


X  100  = 


xl00  = 


xlOO 


D.l  Shoulder/Neck 
Discomfort  Rating 


D.2  Hand/Wrist/Arm 
Discomfort  Rating 


D.3  Back/Torso 
Discomfort  Rating 


D.4  Legs/Feet 
Discomfort  Rating 


D.5  Head/Eyes 
Discomfort  Rating 
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2.3.2.2.1.2  Process  A  scoring  sheet  is  provided  for  the  technician  to  record  tallies  for  each 
body  zone  based  on  responses  to  questions  (47&48,  50&51,  53&54,  56&57,  and  59&60)  and  use 
of  the  Criteria  Table.  If  the  combination  of  employee  responses  fits  into  one  of  the  shaded 
categories  on  the  Criteria  Table,  the  technician  makes  a  tally  mark  in  the  Tally  Box.  The  total 
number  of  tallies  is  used  to  determine  a  Discomfort  Rating  of  Low,  Medium,  or  High  for  each 
body  zone. 

The  Discomfort  Ratings  are  used  to  calculate  the  Survey  Priority  Rank  for  the  shop. 

2.3.2.2.2  General  Questions  Score 

The  scoring  sheet  used  to  calculate  the  General  Questions  Score  is  shown  in  Figure  2.12. 
2.3.2.2.2.1  Rationale  A  separate  score  is  calculated  for  Q61-Q65.  The  Scores  are: 

•  Health  Care  Provider  Score, 

•  Recovery  Time  Score, 

•  Activity  Interruption  Score, 

•  Previous  Diagnosis  Score,  and 

•  Contributing  Factors  Score. 

Responses  averaged  across  the  entire  shop  provide  Public  Health  with  information  against  which 
the  injuiy /illness  documentation  may  be  compared.  For  example,  if  the  Health  Care  Provider 
Score  is  relatively  high  and  there  are  no  recorded  injuries/illnesses  from  the  group,  this  may 
indicate  that  symptoms  of  WMDs  are  being  treated  but  are  not  being  reported.  This  provides 
Public  Health  and/or  the  EWG  with  an  opportunity  to  identify  what  the  potential  sources  of  any 
problems  may  be  (Q64  and  Q65  may  provide  additional  insight). 

For  example,  a  High  Health  Care  Provider  Score,  Low  Previous  Diagnosis  and  Contributing 
Factors  Scores,  and  Low  Risk  Factor  Ratings  may  indicate  employees  are  experiencing  problems 
due  to  outside-work  activities.  Also,  if  a  High  Health  Care  Provider  Score,  Low  Previous 
Diagnosis  and  Contributing  Factors  Scores,  and  High  Risk  Factor  Ratings  are  identified,  this 
could  indicate  that  employees  are  experiencing  problems  that  may  be  work-related  yet  aren’t 
being  reported  as  work-related. 
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Figure  2.12  Scoring  Sheet  -  General  Questions 


Part  II,  Continued 

E  -  General  Questions  (Questions  61  -  65) 


Step  I  Step  2 


Look  at  question  61  and  tally  only 
the  "yes"  answers  in  the  tally  box 
for  that  question.  Count  and  write 
the  total  in  the  total  box. 

Write  the  total  in  the  Health  Care  Provider  Visit  score  box. 

Question  61  Tally  Box 

E.: 

1  Health  Care  Provider 
Visit  Score 

Total 

Step  I 

Step  2 

Step  3 

Step  4 

Look  at  each  question  and  tally 
only  the  "yes"  answers  in  the  tally 
box  for  that  question.  Count  and 
write  the  total  in  the  Total  box. 

Divide  the  total  tallies 
for  that  question  by  the 
number  of  surveys. 

Multiply  that  number 
by  100  to  get  the 
percentage. 

Write  the  shop  percentage  in  the  ; 
box  provided. 

Question  62  Tally  Box 

E.2  Recovery  Time 

,  ~  Score 

number  of 

surveys 

Total 

-f  =  xl00  = 

% 

Question  63  Tally  Box 

E.3  Activity  Interruption 
Score 

number  of 
surveys 

Total 

H-  =  xl00  = 

% 

Question  64  Tally  Box 

E.4  Previous  Diagnosis 
Score 

number  of 
surveys 

Total 

-  =  xl00  = 

% 

Question  65  Tally  Box 

1 

E.5  Contributing  Factors 
Score 

number  of 
surveys 

1 - 1 

Total 


xl00  = 


% 


These  scores  are  provided  to  enable  Public  Health  and  the  EWG  to  interpret  the  Survey  Priority 
Rank  and  other  Scores  to  determine  the  appropriate  follow-up  action  for  EPRA-designated 
shops. 

2.3.2.2.2.2  Process  Again,  a  scoring  sheet  is  provided.  The  Health  Care  Provider  Score  is 
determined  by  coimting  the  total  number  of  “yes”  responses  from  the  shop.  The  goal  is  to 
identify  the  total  number  of  employees  who  have  sought  medical  treatment. 

For  each  of  the  other  scores,  the  number  of  “yes”  responses  is  tallied  and  averaged  across  the 
shop.  These  scores  are  expressed  as  percentages. 

The  General  Questions  Scores  are  not  used  in  calculating  the  Survey  Priority  Rank  for  the  shop. 

2.3.2.3  Part  III;  Work  Content 

Responses  to  the  Part  III,  Work  Content  section  of  the  Survey  are  tallied  for  each  “Type  of 
Work”  that  employees  perform  on  a  routine  basis.  Tasks  that  are  reported  as  routine  by  at  least 
20%  of  shop  employees  are  listed  on  the  Summary  Report.  Public  Health  and  the  EWG  can  use 
this  information  as  an  initial  list  for  identifying  homogeneous  exposure  groups. 

The  Work  Content  information  is  not  used  in  calculating  the  Survey  Priority  Rank  for  the  shop. 

2.3.2.4  Part  IV:  Process  Improvement  Opportunities 

Responses  to  the  Part  IV,  Process  Improvement  Opportunities  section  of  the  Survey  are  not 
scored. 

The  responses,  however,  are  recorded  on  the  Summary  Sheet  for  the  shop  and  may  include  a 
listing  or  a  summary  description  of  hand  or  power  tools,  specific  pieces  of  equipment,  or  specific 
tasks  that  employees  think  may  be  difficult  to  work  with  or  to  perform.  Parts  III  and  IV  are  not 
redundant.  For  example,  the  employee  who  marks  “drilling”  (Q79)  as  a  routine  task  in  Part  III 
makes  no  indication  of  task  difficulty.  However,  the  employee  who  writes  in  response  to  Part 
IV-Q2  that,  “drilling  out  rivets”  is  a  task  which  requires  the  most  effort,  gives  Public  Health 
potentially  highly  valuable  information  regarding  which  task(s)  may  be  exposing  shop 
employees  to  ergonomic  risk  factors  and/or  causing  discomfort.  Employees  may  also  provide 
Public  Health  with  direct  suggestions  on  how  to  improve  work  in  the  shop. 

The  technician  is  encouraged  to  write  down  every  response  to  Part  IV  on  the  Summary  Report 
and  consider  the  responses  as  a  basis  for  problem-solving. 
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Figure  2.13  Summary  Report  -  Page  1 


SUMMARY  REPORT 


ERPA  Status: 

Priority  Rahking: 

Date: 

Date: 

Workplace  Identifier: 

Base: 

Organization: 

Workplace: 

Bldg./Location: 

Room/Area 

AFSC: 

Civilian  Job  Series: 

Shop  Supervisor: 

Duty  Phone: 

Office  Symbol: 

Stepl 


Write  in  the  Risk  Factor  Rating 
forPaki,  (questions  1-38,  Scoring 
Sheetpg.1) 


Step  2 


Write  in  the  Discomfort  Rating 
for  Part  n,  (quesstions  46-60, 
Scoring  Sheet  pg.3)  ? 


Step  3 


Look  at  the  "Ranking  Matrix"  below 
and  enter  the  Priority  Score  in  it's 
corresponding  box. 


A.1 


D.l 


Shoulder/Neck  = 


A.2 


D.2 


HandsAVrist/Arms  = 


A.3 


D.3 


Back/Torso  = 


A.4 


D.4 


Legs/Feet  = 


A.5 


Head/Eye  = 


Ranking 

Matrix 


Ranking  Matrix  for 
Priority  Score 

Discomfort  High 

Discomfort  Medium 

Discomfort  Low 

Risk  Factor  High 

9 

7 

Risk  Factor 

Medium 

8 

5 

Risk  Factor  Low 

6 

Select  the  HIGHEST  score 
for  any  body  part  from  Step  3 
and  enter  —> 


Survey 

Priority 

Rank: 
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2.3.2.5  Summary  Report  -  Survey  Priority  Rank 


The  Summary  Report  consists  of  three  pages. 

2.3.2.5.1  Page  1  The  first  page  (Figure  2.13)  is  used  to  identify  the  shop  and  to  calculate  the 
Survey  Priority  Rank  for  the  shop. 

2.3.2.5.1.1  Rationale  The  Survey  Priority  Rank  for  the  shop  is  determined  by  combining  the 
percentage  of  employees  reporting  discomfort  (refer  back  to  the  Discomfort  Rating  discussion) 
and  the  percentage  of  employees  exposed  to  ergonomic  risk  factors  (refer  back  to  the  Risk  Factor 
Rating  discussion).  The  highest  priorities  are  given  to  shops  (where  the  body  zones  in  which 
both  discomfort  and  risk  factors  are  present).  For  example,  if  a  High  Discomfort  Rating  and  a 
High  Risk  Factor  Rating  were  determined  for  the  shoulder/neck  area,  that  body  part  would 
receive  a  higher  (9)  Priority  Score.  If  a  Low  Discomfort  Rating  and  a  High  Risk  Rating  were 
determined  for  the  shoulder/neck  area,  that  body  part  would  receive  a  lower  (4)  Priority  Score. 

In  the  Priority  Matrix  Table,  the  Discomfort  Ratings  are  weighted  more  heavily  than  the  Risk 
Factor  Ratings.  The  design  of  this  Matrix  was  established  to  reflect  the  Air  Force  philosophy 
that  discomfort  may  be  a  stronger  predictor  of  WMD  than  risk  factor  exposure. 

A  Priority  Rank  of  5  or  higher  should  be  considered  as  EPRA  status. 

2.3.2.5.1.2  Process  Priority  Scores  are  determined  for  each  body  zone  by  transferring 
Discomfort  Ratings  and  Risk  Factor  Ratings  to  the  Summary  page.  The  Ranking  Matrix  is  then 
used  to  identify  the  Priority  Score.  The  process  is  repeated  for  each  of  the  body  zones.  The 
highest  Priority  Score  for  any  body  zone  becomes  the  Survey  Priority  Rank  for  the  shop. 

2.3.2.5.2  Page  2  The  second  page  (Figure  2.14)  is  used  to  summarize  the  Organizational 
Rating,  Physical  Effort  Factor,  and  General  Questions  scores,  as  well  the  results  from  Part  III  of 
the  Survey. 

2.3.2.5.2.1  Rationale  Much  of  the  rationale  behind  the  use  of  the  Organizational  Rating  (item 
B),  Physical  Effort  Factor  (item  C),  General  Questions  scores  (items  E.1-E.5),  and  results  from 
Part  III  (item  F)  was  previously  discussed.  The  combined  information  is  used  to  add  depth  to  the 
Survey  Priority  Rank.  The  information  enables  Public  Health  and  the  EWG  to  interpret  the 
Survey  Priority  Rank,  which  reflects  only  the  Discomfort  Rating  and  Risk  Factor  Rating,  based 
on  all  of  the  factors  to  which  employees  in  a  shop  may  be  exposed. 

All  of  the  information  must  be  considered  when  making  conclusions  and  recommendations  for 
the  shop. 
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Figure  2.14  Summary  Report  -  Page  2 


Step  4 


B.  Enter  Organizational  Rating; 
(Questions  39-44,  Scoring  Sheet  pg. 

Comments: 

Step  5 

C.  Enter  Physical  Effort  Factor 

Comments: 

Score:  (Question  45,  Scoring 

Sheet  pg.2) 

Step  6 

E.  Enter  the  score  for  each  of  the  General  Questions:  (Questions  6 1  -65,  Scoring  Sheet  pg.  4) 

E.1  Health  Care  Provider  Score 

Comments: 

% 

E.2  Recovery  Time  Score 

Comments: 

% 

E.3  Activity  Interruption  Score 

Comments: 

% 

E.4  Previous  Diagnosis  Score 

Comments: 

% 

E.5  Contributing  Factors  Score 

Comments: 

% 

1  Step  7 

F.  List  below  each  of  the  routine  types  of  \vork  which  had  shop  percentage  scores  over  20%.  (Items  66-122,  scoring 

sheet  page  5) 

Type  of  Work 

%  Type  of  Work  % 

2.3.2.5.2.2  Process  Scores  for  Organizational  Rating,  Physical  Effort  Factor,  General 
Questions,  and  Work  Content  are  transferred  from  the  tally  sheets  to  the  Siramiary  Sheet.  The 
technician  is  encouraged  to  provide  an  interpretation  of  the  ratings/scores  in  the  Comments 
column. 

From  the  Work  Content  section  of  the  Scoring  Sheets,  the  technician  generates  a  list  of  routine 
tasks  that  were  identified  by  at  least  20%  of  shop  employees 

2.3.2.5.3  Page  3  The  third  page  (Figure  2.15)  contains  the  remaining  Scoring  Summary 
information. 

2.3.2.5.3.1  Rationale  Three  types  of  information  are  provided  on  page  3  of  the  Summary 
Report:  Potential  Concerns,  Potential  Improvement  Opportunities,  and  Injury/Illness  Data. 
Conclusions  and  Recommendations  are  based  on  a  consideration  of  the  Survey  Priority  Rank  and 
the  balance  of  information  on  Summary  Report  pages  2  and  3. 

2.3.2.5.3.2  Process  From  Part  IV  of  the  Survey,  the  technician  generates  a  list  or  descriptive 
paragraph  of  potential  ergonomic  concerns  (e.g.,  tasks,  tools,  equipment,  etc.)  and/or 
improvement  opportunities. 

From  a  review  of  the  AF  Form  190s,  illnesses/WMDs  that  have  been  reported  in  the  shop  for  the 
past  three  years  are  listed  in  the  Comments  column.  Public  Health  is  encouraged  to  copy  the 
relevant  Form  1 90s  and  attach  the  forms  to  the  Summary  Report. 

The  Conclusions  and  Recommendations  Summary  is  to  be  completed  by  the  EWG.  The  EWG 
will  make  a  determination  of  EPRA  status  (or  not)  based  on  the  Survey  Priority  Rank,  the  ratings 
and  scores  for  the  other  factors,  and  the  interpretation  provided  by  Public  Health. 

The  Conclusions  and  Recommendations  Summary  should  also  indicate  the  intended  follow-up 
action  for  the  shop. 

This  information  enables  Bioenvironmental  Engineering  Services  to  proceed  with  a  Level  1 
evaluation,  if  appropriate. 
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Figure  2.15  Summary  Report  -  Page  3 


Step  8 

Review  Part  IV  (Questions  1-3)  to 
identify  tasks,  tools,  equipment,  etc., 
that  employees  listed  as  potential 
concerns.  Comment  as  appropriate. 

Comments: 

Review  Part  IV  (Question  4)  to 
identify  potential  improvement 

opportunities.  .  Comment  as 

appropriate. 

Comments: 

Step  9 

Injury/Illness  Data:  Review  -  ;  the 
injury/illness  history  from  this  shop. 
Attach  inform  and  comment  as 

appropriate. 

Comments: 

Step  10 

Conclusions  /  Recommendations  Summary 


Shop  Status  Recommendations  for  follow-up: 
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3.0  SURVEY  TESTING  AND  VALIDATION  PROCESS 


3.1  Overview  of  Testing  and  Validation  Process 
The  purpose  of  the  validation  process  is  to: 

•  establish  the  strengths  and  limitations  of  the  Survey;  and 

•  identify  the  need  for  changes  based  on  quantitative  information. 

A  variety  of  techniques  have  been  used  by  researchers  to  validate  ergonomic  assessment  tools.  A 
review  of  techniques  is  presented  in  Table  3.1. 

Table  3.1  Techniques  Used  to  Validate  Ergonomics  Assessment  Tools 


Reference 

Validation  Techniques 

Keyserling  et  al,  1993  [2] 

•  Novice  checklist  vs.  Expert  detailed 
analysis  (concurrent  validity) 

Stetson  etal,  1991  [27] 

•  Inter-rater  agreement  (reliability) 

Lifshitz  &  Armstrong, 
1986  [13] 

•  Checklist  score  vs.  Incidence  rate 
(predictive  validity) 

Kemmlert,  1994  [12] 

•  Comparison  with  another  scale  (concurrent 
validity) 

•  Items  based  on  literature  (content  validity) 

•  Inter-rater  agreement  (reliability) 

Engkvist  et  al,  1995  [28] 

•  Inter-rater  agreement  (reliability) 

McAtammey,  1993  [14] 

•  Checklist  score  vs.  discomfort  (predictive 
validity) 

•  Inter-rater  agreement  (reliability) 

Cole,  1995  [1] 

•  Test/re-test  consistency  (reliability) 

•  Checklist  score  vs.  job  type  (concurrent 
validity) 

Silverstein  et  al,  1991  [29] 

•  Student  score  vs.  expert  score,  same 
checklist  (reliability,  usability) 

Baron  et  al,  1996  [8] 

•  Test/re-test  consistencies,  inter-item 

correlation  (reliability) 

•  Assessment  results  vs.  physical  exam 
(predictive  validity) 

3-1 

88 


Based  on  the  techniques  reported  in  the  literature,  the  process  used  to  validate  the  effectiveness 
of  the  Survey  was  comprised  of  three  distinct  steps: 

•  usability  testing; 

•  test/re-test  reproducibility;  and 

•  concurrent  validity  testing. 

3.2  Methods 

3.2.1  Usability  Testing 

Usability  testing  was  performed  to  insure  that  the  end  users  would  be  able  to  administer  the 
Survey  as  it  was  designed  and  according  to  Air  Force  objectives.  Usability  testing  focused  on 
both  the  sxuwey  questions  and  the  scoring  procedures.  The  usability  testing  was  performed  at 
Malmstrom  AFB.  A  group  of  25  Air  Force  persoimel  from  a  variety  of  shops  completed  the 
Survey  and  participated  in  a  focus  group.  During  the  focus  group,  employees  commented  on  the 
clarity  and  appropriateness  of  questions  to  Air  Force  operations.  The  Public  Health  staff 
completed  the  scoring  process  and  commented  on  the  scoring  methodology. 

3.2.2  Reproducibility  Testing 

The  purpose  of  reproducibility  testing  is  to  determine  how  consistently  a  survey  tool  produces 
the  same  results.  Reproducibility  testing  generally  determines  the  upper  limits  of  the 
effectiveness  of  a  tool.  A  tool  can  not  be  better  than  the  degree  to  which  it  consistently  obtains 
the  same  results. 

Reproducibility  testing  is  generally  performed  one  of  several  ways.  Inter-rater  reproducibility 
examines  the  similarity  of  results  obtained  by  different  raters.  For  instance,  if  two  people  use  a 
tape  measure  to  determine  the  length  of  an  object,  the  degree  to  which  they  agree  on  the  length 
demonstrates  inter-rater  reproducibility.  Another  form  of  reproducibility  is  the  degree  to  which  a 
person  provides  the  same  response  at  different  times.  This  is  referred  to  as  test/re-test 
reproducibility.  Test/re-test  reproducibility  is  the  preferred  method  for  self-reporting  tools  since 
each  person’s  actual  experience,  as  well  as  perception,  may  vary. 

Test/re-test  reproducibility  was  conducted  for  the  Survey  since  it  is  a  self-reporting  tool. 

3.2.2. 1  Study  Group 

The  contractor  performed  a  two-week  pilot  test/re-test  study  at  Peterson  AFB.  The  primary 
purpose  of  the  pilot  study  was  to  determine  if  the  participants  could  consistently  answer  the 
Survey  questions.  A  two-week  interval  was  chosen  to  ensure  sufficient  delay  so  that  participants 
would  not  remember  their  responses  while  maintaining  a  short  enough  interval  that  the 
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participant’s  job  demands  and  discomfort  would  remain  constant  [31].  In  order  to  determine  a 
sample  size  estimate,  the  statistical  power  for  a  Pearson  correlation  was  utilized  with  a  desired 
power  of  .95  (assuming  a  correlation  of  .5  and  alpha  of  .05).  Test  planning  called  for  50 
participants  from  shops  with  known  or  suspected  ergonomic  hazards  and  additional  participants 
from  shops  known  to  have  minimal  risk  to  serve  as  a  control  group.  As  many  as  75  to  100 
participants  were  expected  for  the  initial  Survey  administration.  Fifty  employees  from  five  shops 
participated.  Forty  participants  returned  for  the  second  Survey  administration.  Twenty-seven  of 
the  40  participants  were  from  shops  with  known  or  suspected  ergonomic  hazards.  The  data  for 
the  participants  who  did  not  participate  in  the  second  Survey  administration  was  removed  before 
conducting  analyses. 

Participants  were  assured  of  the  anonymous  nature  of  the  survey.  The  tracking  of  surveys 
between  sessions  was  conducted  using  random  code  numbers  knovra  only  by  the  individual 
participants.  This  anonymity  was  important  to  facilitate  accurate  responding  and  to  produce  an 
environment  similar  to  the  enviroiunent  expected  during  actual  use  of  the  survey  tool. 

Due  to  budgetary  constraints,  the  nxunber  of  shops  that  participated  in  the  pilot  investigation  was 
limited.  The  shops  did  represent  a  variety  of  Air  Force  tasks;  however,  the  sample  size  was  not 
sufficient  to  capture  the  full  diversity  of  tasks  found  throughout  Air  Force  operations.  As  a 
result,  the  personnel  who  took  part  in  reproducibility  testing  are  not  considered  to  be  completely 
representative  of  all  personnel  who  would  be  completing  the  Survey  after  its  use  is  adopted. 

The  combination  of  the  small  sample  size  of  persons  from  shops  with  ergonomic  hazards  and  the 
non-representative  nature  of  the  sample  group  resulted  in  very  low  response  rates  to  some 
questions.  As  a  result,  14  of  the  38  risk  factor  questions  may  not  have  had  sufficient  response 
rates  of  risk  factor  presence  to  make  any  definitive  reproducibility  statements.  It  is  possible  that 
the  tasks  performed  in  these  shops  did  not  involve  the  risk  factors  depicted  in  these  14  questions. 
However,  since  the  questions  were  obtained  from  validated  sources  and  were  either  incorporated 
into  the  Survey  in  their  original  form  or  modified  slightly,  it  is  expected  that  the  questions 
remain  reproducible.  The  questions  with  low  response  rates  are  listed  in  the  reproducibility 
results  table  without  accompanying  statistics  but  with  a  note  of  the  number  of  responses  greater 
than  2  hours.  The  reproducibility  results  should  be  interpreted  cautiously  in  consideration  of  the 
non-representative  nature  of  the  sample.  Table  3.2  lists  the  number  of  participants  by  shop. 
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Table  3.2  Shop  Participants  for  Test/Re-Test  Evaluation 


Shop 

Number  of  Participants 

Bioenvironmental  Engineering 
Services  and  Public  Health  (Control 
shops) 

13 

Dental  Lab 

12 

Falcon  AFB  (Training  Group) 

7 

Structural  Maintenance  (Heavy  duty 
maintenance/repair  aircraft) 

5 

Survival  Equipment  Repair 

3 

3.2.2.2  Statistical  Analyses 

A  variety  of  statistical  methods  have  been  reported  in  the  literature  for  measuring  reproducibility, 
including;  percent  agreement,  correlation’s,  coefficient  of  concordance,  chi-square,  and  Kappa. 
Meister  [30]  provides  an  overview  of  each  of  these  reproducibility  methods  with  comments 
regarding  their  practical  utility  for  behavioral  analysis  methods.  Meister  concludes  that  each 
method  has  certain  limitations  and  that  no  single  measure  of  reproducibility  is  agreed  upon. 

The  selection  of  the  Kappa  statistic  was  based  on  its  relative  ease  of  interpretation  and 
comparability  to  the  reproducibility  results  obtained  by  Wiktorian  et  al  [31].  These  factors  make 
the  Kappa  a  good  choice  for  reporting  the  reproducibility  results. 

In  order  to  compare  the  two  administrations  of  the  Survey,  a  weighted  Kappa  was  performed. 
(Cohen,  1960  [32];  Fleiss  &  Cohen,  1973  [33];  Bartko  &  Carpenter,  1976  [34]).  The  weighted 
Kappa  compared  the  survey  responses  for  each  Survey  administration.  Since  it  is  expected  that  a 
certain  amount  of  agreement  would  occur  by  chance,  such  as  having  50%  correct  on  a  true/false 
test,  the  Kappa  statistic  reports  agreement  after  chance  has  been  removed.  A  Kappa  value  can 
be  interpreted  as  a  percent  of  agreement.  For  instance,  a  Kappa  of.  75  indicates  an  agreement 
rate  of  75%)  after  chance  has  been  removed. 

A  95%  confidence  interval  is  reported  for  each  Kappa.  Because  statistical  testing  uses  a  smaller 
sample  to  predict  the  actual  results  of  a  larger  population,  the  actual  population  results  may  be 
different  than  the  sample  results.  The  confidence  interval  reports  a  range  in  which  the  population 
results  could  reasonably  be  expected  to  fall.  The  confidence  interval  is  affected  by  sample  size 
and  variability.  A  small  sample  size,  such  as  the  Survey  reproducibility  testing,  results  in  a 
larger  range  for  the  confidence  interval.  When  the  lower  bounds  of  the  confidence  interval  were 
below  chance  agreement  (Kappa  <  0.0),  it  suggests  that  the  agreement  reported  by  the  Kappa 
obtained  from  the  test  group  may  not  reflect  an  agreement  above  the  chance  level  for  the 
population. 
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In  terms  of  the  true/false  test  scenario  presented  previously,  if  a  test  had  1 00  questions  it  would 
be  difficult  to  predict  the  whole  test  score  based  on  knowledge  that  seven  of  ten  questions  were 
answered  correctly.  The  prediction  of  whole  test  scores  would  improve  with  knowledge  that  35 
of  50  questions  were  answered  correctly.  Although  the  proportion  of  agreements  (test  answers 
with  correct  answers)  is  the  same  in  each  case,  in  the  second  scenario  more  accurate  estimates 
can  be  made.  A  confidence  interval  is  a  numerical  reporting  of  this  prediction  accuracy.  As  a 
means  of  better  understanding  Kappa  values,  assuming  an  equal  overall  ratio  of  true  and  false 
responses,  the  Kappa  value  for  each  of  the  above  scenarios  would  be  .40. 

The  Kappa  values  were  calculated  for  each  individual  Survey  question,  as  well  as  for  the  body 
area  section  scores  for  the  Job  Factors  and  Discomfort  Factors  sections  of  the  survey.  When 
responses  were  not  dichotomous,  a  weighted  Kappa,  with  squared  deviations  from  agreement  as 
weights,  was  used  as  suggested  by  Maclure  and  Willett,  1987  [35].  The  contractor  used  SAS 
version  6.11  for  Windows  to  complete  the  analyses.  Table  3.3  provides  a  summary  of  the 
guidelines  which  were  used  to  interpret  the  Kappa  values. 

Table  3.3  Guidelines  for  Interpreting  Kappa  Values 


Kappa  Values 

Interpretation 

n 

.81  to  1.0 

Almost  Perfect 

.61  to  .80 

Substantial 

.41  to  .60 

Moderate 

.21  to  .40 

Fair 

0.0  to  .20 

Slight 

<0.0 

Poor 

The  interpretations  provided  in  Table  3.3  are  consistent  with  those  suggested  by  Landis  and 
Koch  (1977)  [36].  In  order  to  conclude  that  modifications  to  questions  did  not  substantially  alter 
previous  validations  of  these  questions,  results  were  expected  to  be  similar  to  those  obtained  by 
Wiktorian  et  al  [31].  Because  of  the  small  sample  size,  the  findings  are  considered  indicative  of 
similarity  to  previous  research  findings  rather  than  definitive  statements  of  reproducibility.  The 
Kappa  values  obtained  by  Wiktorian  et  al  [31]  for  working  postures  and  material  handling 
descriptors  generally  ranged  between  .35  and  .50  with  the  lower  confidence  interval  (95%) 
typically  above  .25  and  the  upper  confidence  interval  below  .55.  On  the  combined  basis  of  the 
descriptive  interpretations  [36]  and  the  previous  findings  [31],  the  obtained  values  of  Kappa  were 
expected  to  fall  in  the  “moderate”  range  suggesting  that  the  modified  questions  retained  similar 
reproducibility  to  the  original  questions.  When  Kappa  values  were  below  .40,  or  when  the  lower 
range  of  the  confidence  interval  was  below  .20,  it  was  determined  that  the  reproducibility  of 
these  questions  may  be  lower  than  the  initial  questions.  Explanations  of  the  lower 
reproducibility  values  were  investigated  and,  when  needed,  modifications  to  the  question  were 
considered. 
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Since  the  Organizational  Factors  used  a  5-point  Likert  scale  for  responses,  several  analyses  were 
conducted  to  describe  the  data.  The  weighted  Kappa  was  performed  on  the  5  point  responses  for 
consistency  with  the  other  Factors.  Because  the  scoring  is  based  on  the  occurrence  of  agree  or 
strongly  agree  (with  no  distinction  between  these  two  responses)  the  data  was  split  between  agree 
and  the  combination  of  neutral  and  disagrees.  A  Kappa  was  performed  comparing  these  two 
levels  and  reported  as  a  “2  level  Kappa.”  Likert  scales  can  generally  be  considered  as  interval 
type  data  (Meister)  and,  as  such,  the  Pearson  correlation  is  presented  as  a  measure  of  association. 

Missing  values  (non-responses)  for  individual  questions  in  the  Job  Factors  questions  were  eoded 
the  same  as  “zero  hours  daily”  responses.  Since  a  “never”  category  did  not  exist,  this  coding  is 
consistent  with  the  manner  in  which  these  responses  would  be  scored  by  teehnieians  when 
administering  the  survey.  While  treating  a  blank  response  as  a  non-existence  of  the  risk  faetor 
may  lead  to  a  statement  of  lower  ergonomic  risk  within  a  shop,  it  appears  to  be  the  most 
eonsistent  method  for  selecting  a  score  for  the  survey.  Since  this  was  a  pilot  study  with  a  low 
sample  size,  eliminating  these  subjects  from  the  analysis  would  have  greatly  reduced  the  ability 
to  obtain  or  interpret  results.  The  Job  Factors  questions  have  been  modified  to  include  a  “never” 
response  which  should  greatly  reduce  the  number  of  blank  survey  responses. 

A  post  hoc  comparison  was  conducted  on  the  shop  level  scores  using  a  Spearman  rank  order 
correlation  as  a  pilot  investigation  of  the  reproducibility  of  shop  level  scores.  The  Spearman 
correlation  was  selected  based  on  the  rankings  nature  of  the  data.  It  is  assumed  that  the  Priority 
Ranks  provided  as  output  from  this  tool  provide  information  of  an  ordinal  nature  rather  than  an 
interval  nature.  Existing  research  provides  minimal  insight  into  reproducibility  at  this  level. 
While  the  sample  size  of  five  shops  resulted  in  a  relatively  low  power,  the  results  are  presented 
as  an  initial  indication  of  shop  level  reproducibility. 

3.2.3  Validity  Testing 

Once  the  reproducibility  of  a  tool  has  been  established,  the  meaning  of  the  answers  needs  to  be 
evaluated.  In  research  terms  this  is  called  validity.  There  are  several  types  of  validity.  The  most 
stringent  is  predictive  validity.  A  college  may  base  its  admission  decisions  largely  on  previous 
GPA  because  research  and  experience  has  shown  that  the  best  predictor  of  future  academic 
performance  is  past  academic  performance.  This  is  an  example  of  predictive  validity.  Predictive 
validity  is  rare  in  ergonomic  assessment  tools. 

A  more  common  validity  measure  in  ergonomics  is  concurrent  validity.  Concurrent  validity  uses 
one  type  of  measure  to  validate  another  measure.  For  instance,  oxygen  consumption  might  be 
measured  to  validate  a  metabolic  expenditure  model.  This  is  the  type  of  validation  that  was 
performed  on  the  Survey.  A  Masters  degree-level  (M.S.  in  Industrial  Engineering)  Ergonomist 
was  selected  as  the  “Ergonomist  Expert”  based  on  his  previous  experience  with  the  types  of  work 
found  throughout  the  Air  Force.  The  results  of  the  Ergonomist’s  assessment  of  shop  activities 
would  be  compared  against  the  results  obtained  from  the  Survey  method.  The  similarity  of 
rankings  would  determine  how  much  the  two  measures  agree  with  each  other.  An  extensive 
review  of  the  research  literature  did  not  provide  any  previous  studies  of  this  specific  relationship 
to  predict  what  degree  of  correlation  to  antieipate  between  an  Ergonomist  ranking  of  shops  and  a 
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Survey  ranking  of  shops.  The  contractor  estimated  the  correlation  between  .5  and  .7  prior  to 
conducting  validity  testing  based  on  data  regarding  inter-rater  agreement  at  the  job  level.  This 
estimate  was  used  to  establish  sample  size  requirement  and  to  specify  the  objectives  of  the 
Survey. 

3.2.3.1  Procedures  (e.g..  Ergonomist  analysis  vs.  Survey  results) 

The  Ergonomist  visited  3 1  shops  (26  shops  at  Patrick  AFB  and  5  shops  at  Cape  Cod  AS)  two 
weeks  prior  to  the  administration  of  the  Survey.  The  two-week  time  period  was  established  to 
minimize  the  potential  that  employees  would  respond  to  a  Survey  question  based  on  discussions 
that  may  have  occurred  during  the  Ergonomist’s  shop  visits  and  to  minimize  the  possibility  of 
significant  changes  in  the  tasks  performed  by  shop  personnel.  The  Ergonomist  was  intentionally 
kept  blind  to  the  rationale,  development,  content,  and  scoring  of  the  Survey.  During  the  shop 
visits,  the  Ergonomist  identified  himself  as  a  “safety  contractor”  to  limit  the  potential  for  biasing 
the  subsequent  Survey  administration. 

While  visiting  the  shops,  the  Ergonomist  obtained  an  overview  of  activities  within  each  shop, 
completed  an  independent  ergonomic  assessment  on  selected  representative  tasks,  and  reviewed 
the  injury  and  illness  records  for  the  shop.  The  Ergonomist  was  instructed  by  the  Survey 
designers  to  provide  (1)  a  risk  statement  for  each  of  the  five  body  areas  used  in  the  Survey,  (2)  an 
overall  risk  rating  for  the  shop,  and  (3)  a  relative  ranking  of  shops  (e.g.  list  of  shops  firom  greatest 
to  least  amount  of  risk  factor  exposure. 

The  contractor  instructed  base  personnel  from  Patrick  AFB  and  Peterson  AFB  (in  cooperation 
with  Cape  Cod  AS)  in  the  Survey  administration.  The  importance  of  an  80%  response  rate  from 
shops  was  emphasized  verbally  and  repeated  in  the  written  Survey  administration  instructions. 
The  contractor  participated  in  the  administration  and  scoring  of  the  Survey  at  two  Patrick  AFB 
shops  in  order  to  demonstrate  the  complete  process.  Survey  administration  and  scoring  for  the 
remaining  shops  was  conducted  by  base  personnel.  As  planned,  the  Survey  was  administered  at 
least  two  weeks  after  the  Ergonomist’s  shop  visits.  Shop  scoring  was  performed  by  Air  Force 
personnel,  and  the  results  were  provided  to  the  contractor  for  analysis.  A  response  rate  of  at  least 
80  percent  was  obtained  from  18  of  the  31  shops.  However,  in  13  shops  the  response  rates  were 
below  80%. 

3.2.3. 2  Statistical  Analyses 

The  body  area  Risk  Factor  Ratings,  Priority  Scores,  and  Survey  Priority  Rank  obtained  using  the 
Survey  were  compared  to  the  Ergonomist’s  findings  using  a  Spearman  rank-order  correlation. 
The  agreement  between  the  Survey  body  area  Risk  Factor  Ratings  and  the  Ergonomist’s  body 
area  risk  ratings  were  calculated  using  a  weighted  Kappa. 
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3.3  Results 


3.3.1  Usability  Testing 

The  results  of  the  usability  testing  indicated  that  the  length  of  time  to  administer  and  score  the 
survey  exceeded  the  project  goals.  The  completion  of  the  Survey  administration  required  one 
hour  and  20  minutes.  The  scoring  requirements  for  a  shop  of  25  people  were  extrapolated  from 
smaller  data  sets.  When  it  became  apparent  that  the  scoring  process  was  taking  significantly 
more  time  than  was  allocated,  the  number  of  Surveys  scored  by  each  person  was  reduced  to  five 
or  fewer  Surveys  per  person.  An  extrapolation  of  the  data  indicated  that  scoring  a  shop  of  25 
people  would  take  between  6  and  8  hours. 

3.3.1. 1  Survey  Design  Feedback 

At  the  time  of  the  usability  testing,  the  alpha-version  Survey  design  included  separate  sections 
for  administrative,  warehouse,  assembly,  and  maintenance/inspection  work  types.  Participants 
indicated  that  they  did  not  know  which  sections  (e.g.,  one?,  more  than  one?)  they  were  supposed 
to  complete.  Also,  some  participants  felt  that  if  they  were  exposed  to  a  risk  factor  for  2  hours 
and  they  responded  in  the  “0-2  hours”  category,  they  would  be  counted  the  same  way  as  a  person 
who  was  never  exposed  to  a  risk  factor.  Conversely,  participants  who  were  never  exposed  to  a 
risk  factor  felt  that  by  responding  in  the  “0-2  hours”  category,  they  would  be  overstating  their 
exposure. 

3.3.1. 2  Scoring  Process  Design  Feedback 

The  alpha-version  Survey  scoring  included  relative  importance  weightings  for  risk  factors. 
During  the  scoring.  Public  Health  representatives  commented  that  the  scoring  and  weighting 
processes  were  confixsing  and  too  time  consuming. 

3.3. 1.3  General  Comments  on  Administration 

The  users  and  future  administrators  also  commented  that  the  Ergonomics  Overview  was  too  long 
and  that  it  should  be  limited  to  providing  information  only  on  the  purpose  for  completing  the 
survey,  how  the  results  will  be  used,  and  the  survey  instructions. 

3.3.1.4  Changes  Made  to  Improve  Usability 

Based  on  the  feedback,  the  following  changes  were  made: 

•  the  overview  was  reduced  and  limited  to  providing  information  on  purpose,  outcome,  and 
instructions; 

•  the  number  of  questions  was  reduced; 
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•  redundancy  was  eliminated  by  combining  the  general  and  area  specific  questions  into  one  Job 
Factors  section; 

•  a  “never”  response  category  was  added  to  parts  I  and  III  of  the  Survey  as  a  result  of  this 
feedback; 

•  the  amount  of  hand  tallying  was  reduced;  and 

•  weightings  for  individual  risk  factor  questions  was  eliminated  since  the  difficulty  in  scoring 
far  exceeded  the  expected  value  of  weighting  for  discriminating  between  shops. 

One  additional  usability  request  made  by  the  administrators  of  the  Survey  during  reproducibility 
testing  at  Peterson  AFB  was  the  inclusion  of  written  instructions  which  would  allow  the  option 
of  “dropping-off’  the  blank  Surveys  and  asking  shop  personnel  to  complete  the  Surveys  without 
any  introductory  remarks.  This  request,  however,  was  not  adopted  because  of  the  potential  for 
poor  response,  loss  of  completed  Surveys,  etc.,  and  to  avoid  creating  misconceptions  about  the 
purpose  and  use  of  the  Survey  results. 

3.3.2  Reproducibility  Testing 

3.3.2.1  Section  Tally  Results 

During  the  scoring  of  a  shop,  each  survey  is  reviewed  and  a  yes/no  decision  is  made  regarding  a 
tally  for  each  of  the  five  body  zone  Job  Factors  and  Discomfort  Factors  sections.  These  ten  tally 
marks  are  the  only  risk  factor  and  discomfort  information  used  from  the  individual  surveys  to 
establish  shop  scoring.  Agreement  at  this  level  has  a  greater  impact  on  the  shop  scoring  than 
does  the  agreement  to  individual  questions.  Poor  agreement  or  non-significant  results  at  the  tally 
level  demands  greater  attention. 

Table  3.4  shows  the  actual  agreement  rates,  weighted  Kappa  values,  and  95%  confidence 
intervals  for  each  of  the  section  tallies.  Eight  of  the  ten  tally  sections  had  agreements  in  the  fair 
to  good  range.  In  the  tally  sections  for  the  back/torso,  both  the  Job  Factors  and  Discomfort 
Factors  displayed  lower  agreement  rates.  The  low  weighted  agreement  rates  may  be  a  by¬ 
product  of  a  low  number  of  back-intensive  jobs  in  the  sample  group.  Less  than  five  people 
responded  to  questions  in  a  manner  that  would  lead  to  a  tally  for  the  back/torso  in  either  of  the 
two  Survey  administrations.  Ratings  for  leg  discomfort  appear  to  be  more  variable  than  the  other 
body  areas.  The  agreement  rates  (not  chance  corrected)  were  consistently  above  80%.  The  high 
raw  agreement  rates  combined  with  the  lower  Kappa  values  reflect  the  high  degree  of  responses 
in  one  category.  Since  the  chance  agreement  is  high,  the  chance  corrected  agreement  (Kappa)  is 
naturally  lower. 
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Table  3.4  Test/Re-test  Agreement— Section  Tallies 


TaUy 

Raw  % 
Agreement 

Weighted 

Kappa 

95% 

LCI 

95% 

UCI 

Comments 

Shoulder  Risk  Factor  Tally 

80% 

.429 

.099 

.758 

Hand/Wrist  Risk  Factor 
Tally 

88% 

.679 

.935 

Back/Torso  Risk  Factor 
Tally 

90% 

.286 

-.214 

.785 

Low  Kappa  value  may  be  due  in 
part  to  the  limited  number  of 
back-intensive  jobs  in  the  test 
sample 

Legs/Feet  Risk  Factor 

Tally 

80% 

.60 

.364 

.836 

Head/Eyes  Risk  Factor 

Tally 

80% 

.467 

.782 

Shoulder  Discomfort  Tally 

83% 

.573 

.292 

.854 

HandWrist  Discomfort 
Tally 

88% 

.679 

.424 

.935 

Back/Torso  Discomfort 
Tally 

83% 

.364 

-.006 

.733 

Low  Kappa  value  may  be  due  in 
part  to  the  limited  number  of  back 
intensive  jobs  in  the  test  sample 

Legs/Feet  Discomfort  Tally 

95% 

.474 

-.148 

1.095 

The  upper  Cl  exceeds  1.0  due  to 
the  large  variance  of  Kappa. 

Head/Eyes  Discomfort 

Tally 

85% 

.571 

.268 

.875 

3.3.2.2  .Tob  Factor  Question  Results 

Job  Factor  question  results  are  provided  in  Table  3.5.  Due  to  the  nature  of  the  sample 
population,  many  questions  had  low  response  rates  in  the  categories  above  2  hours  per  day.  The 
high  prevalence  of  responses,  over  90%  of  respondents  in  some  cases,  in  the  less  than  2-hour 
category  created  a  situation  where  a  single  response  disagreement  could  greatly  reduce  the  Kappa 
value.  When  a  Kappa  value  could  not  be  computed  due  to  an  uneven  distribution  of  the  data,  no 
figures  are  reported  in  the  table.  When  the  average  number  of  responses  from  the  test  and  re-test 
conditions  was  less  than  five,  the  number  of  responses  is  recorded  in  the  comments  section.  The 
Kappa  values  are  not  reported  when  response  rates  were  less  than  5  to  avoid  drawing 
imwarranted  conclusions. 

Based  on  the  limited  number  of  personnel  and  job  categories  that  were  available  for  the  test,  the 
following  preliminary  interpretation  is  provided.  For  those  questions  with  sufficient  response 
rates,  most  questions  had  sufficient  high  agreement  to  include  confidently.  A  total  of  38 
questions  were  developed.  Eighteen  (47%)  were  tested  using  the  weighted  Kappa  statistic.  Of 
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the  18  tested,  16  (89%)  either  met  or  exceeded  the  goal  of  .40,  furthermore  15  of  the  questions 
had  lower  confidence  intervals  of  .199  or  higher.  The  Kappa  results  suggest  that  the  revised 
questions  had  similar  reproducibility  rates  as  previously  published  results  [31].  The  raw 
agreement  percentages  indicate  that  most  questions  (28/38)  had  agreement  rates  higher  than  80%. 
Therefore,  while  the  sample  size  is  small,  this  result  suggests  that  modifications  to  the  questions 
did  not  have  any  negative  affects  on  the  reproducibility  and  may  have  improved  the 
reproducibility  of  some  questions.  A  much  larger  sample  size  would  be  required  to  firmly 
establish  reproducibility  and  to  determine  if  the  reproducibility  of  some  questions  had  been 
substantially  improved.  Sample  size  is  an  important  contributing  factor  to  a  low  variance  (and 
therefore  confidence  intervals  that  are  close  to  the  actual  score).  The  small  sample  size  in  this 
pilot  test  may  have  contributed  to  large  confidence  intervals,  which  limits  the  ability  to  make 
definitive  statements  regarding  the  reproducibility  of  items  in  this  scale.  The  Kappa  values  and 
raw  agreement  percentages  suggest  a  moderate  level  agreement,  similar  to  existing  scales. 

The  less  than  perfect  Kappa  agreement  should  be  interpreted  in  light  of  previous  research  on 
working  postures  questions  [31]  which  demonstrated  agreement  rates  between  .32  and  .68,  as 
well  as  research  on  self-reporting  in  general  which  indicates  reproducibility  Kappa’s  as  low  as 
.43  for  “have  you  ever  taken  vitamins?”  and  .62  for  “do  you  drink  coffee?”  (Kelly,  Rosenberg, 
Kaufinan  and  Shapiro,  1990  [37]).  Furthermore,  the  raw  agreement  rates  were  consistently 
above  80%,  indicating  a  substantial  agreement  between  testing  sessions. 

Table  3.5  Weighted  Kappa  Statistics  for  Job  Factor  Questions 


Question 

1.  hands  at  or  above  chest  level 

Raw  % 
Agreement 

75% 

Weighted 

IIKipphli 

.584 

95% 

LCI 

.342 

95% 

UCI 

.827 

Comments 

2.  lay  on  back  or  side 

100% 

0  responses  >  2  hrs 

3.  hold  or  carry  materials 

88% 

3  responses  >  2  hrs 

4.  force  or  yank  components 

88% 

2  responses  >  2  hrs 

5.  reach  or  hold  arms  in  front  of 
body 

70% 

.603 

.398 

.807 

6.  neck  is  tipped 

.444 

.220 

.669 

7.  cradle  a  phone 

4  responses  >  2  hrs 

8.  wrists  are  bent 

70% 

.444 

.209 

.679 

9.  apply  pressure  for  more  than  10 
seconds 

80% 

.773 

.627 

.919 

10.  similar  to  clothes  wringing 

78% 

.055 

-.20 

.311 

Question  clarified  for  current 
version 

1 1 .  repetitive  tasks 

70% 

.632 

.440 

.824 

12.  red  marks  on  skin 

83% 

.472 

.159 

.785 
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Table  3.5  Weighted  Kappa  Statistics  for  Job  Factor  Questions  (Contd.) 


Question 


13.  hand  as  a  hammer 


14.  fingers  are  cold 


15.  incentive  or  quota 


16.  tools  vibrate  or  jerk 


17.  throw  or  toss 


18.  twist  forearms 


19.  bulky  gloves 


20.  hand  pressure 


21.  pinch  grip  tightly 


22.  hands  below  knees 


23.  lean  forward  continually 


24.  PPE  restrictive 


25.  repetitive  back  movements 


26.  lifting  twisted  or  quickly 


27.  whole  body  vibration 


28.  1-hand  lift/carry 


29.  lift  bulky  items 


30.  lift  more  than  25  lbs 


31.  kneel  or  squat 


35.  glare 


36.  noise 


37.  vigilance 


38.  lighting 


Raw%  Weighted  95%  95% 

Agreement  Kappa  LCl  ITCI 


93% 


Comments 


2  responses  >  2  hrs 


3  responses  >  2  hrs 


1  response  >  2  hrs 


3  responses  >  2  hrs 


1  response  >  2  hrs 


0  responses  >  2  hrs 


3  responses  >  2  hrs 


Non-square 


1  response  >  2  hrs 


2  responses  >  2  hrs 


Non-square 

2  responses  >  2  hrs 

3  responses  >  2  hrs 


4  responses  >  2  hrs 


32.  apply  foot  pressure 

88% 

.576 

.277 

.875 

33.  feet  off  floor 

83% 

.316 

-.01 

.641 

Illustration  modified 

34.  stand  on  hard  surfaces 

75% 

.840 

.779 

2  responses  >  2  hrs 

3  responses  >  2  hrs 


Based  on  the  results  of  the  reproducibility  testing,  several  questions  were  re-worded  to  improve 
clarity.  An  illustrative  figure  was  added  to  questions  7  and  10.  The  wording  was  modified  for 
question  20.  The  illustration  for  question  33  was  also  modified. 

3 .3.2.3  Organizational  Factors  and  Physical  Effort  Scales  Results 

For  the  Organizational  Factors  questions  (Q39  -  Q44),  a  weighted  Kappa  is  reported  for  the 
entire  5  level  scale  where  possible.  A  dichotomous  response  grouping  was  also  created  for  the 
Organizational  Factors  questions  comparing  the  “agree”  responses  with  the  “neutral”  and 
“disagree”  responses.  This  is  reported  as  a  two-level  Kappa  under  the  comments  column  in 
Table  3-6.  The  results  of  a  Pearson  correlation  are  also  presented  for  the  psycho-social  and  Borg 
scale  questions. 

The  responses  to  the  Organizational  Factor  questions  were  among  the  most  consistent  for  the 
entire  survey.  Six  organizational  factors  questions  were  developed  and  all  six  (100  %)  had 
sufficient  response  rates  to  be  tested.  All  of  the  questions  (100%)  had  Kappa  values  that 
exceeded  the  goal  of  .40  with  lower  confidence  intervals  above  .33. 

The  two-level  Kappa’s  generally  suggested  “substantial”  agreement.  The  Kappa  values  were 
closer  to  the  raw  agreement  data  for  these  questions  because  the  response  were  more  evenly 
distributed  across  categories  (which  produces  a  lower  chance  agreement  rate  correction  in  the 
Kappa  formula).  The  higher  degree  of  reproducibility  in  the  organizational  questions  compared 
to  the  job  factor  questions  may  reflect  the  nature  of  the  questions  (the  questions  may  be  easier  for 
people  to  respond  to)  or  the  more  familiar  response  scale  of  strongly  agree  to  strongly  disagree. 


Table  3.6  Weighted  Kappa  Statistics  for 
Organizational  Factors  and  Physical  Effort  Questions 


Question 

Raw  % 

Agreement 

Weighted  Kappa 

95% 

LCI 

95% 

UCI 

Comments 

39.  unclear  responsibilities 

65% 

.590 

.373 

0.67 

r=. 67/2  level 

Kappa  .68  Cl  .35  to 
1.02 

40.  heavy  workload 

58% 

.624 

.459 

.789 

r=. 74/ 2  level 

Kappa  .70  Cl  .45  to 
.94 

41.  conflicting  demands 

60% 

r  =  .79/  2  level 

Kappa  .73  Cl  .49  to 
.98 

42.  unable  to  get  information 

70% 

r  =  .69/  2  level 

Kappa  .61  Cl  .33  to 
.89 
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Table  3.6  Weighted  Kappa  Statistics  for 
Organizational  Factors  and  Physical  Effort  Questions  (Contd.) 


Question 

Raw% 

Agreement 

Weighted  Kappa 

liiiil 

95% 

UCI 

Comments 

43.  supervisor  feedback 

58% 

r  =  .68/  2  level 

Kappa  .62  Cl  .29  to 
.95 

44.  amount  of  work  interferes  with 

quality 

70% 

.464 

.276 

.653 

r  =  .64/2  level 

Kappa  .63  Cl  .36  to 
.90 

45.  Borg  scale 

r  =  .78/non-square 

3.3.2.4  Discomfort  Factor  and  General  Questions  Results 

The  Discomfort  Factor  questions,  with  the  exception  of  back/torso,  demonstrated  generally 
“moderate”  or  “substantial”  agreement,  with  most  Kappa  values  near  or  above  .60  and  lower 
confidence  intervals  above  .20.  The  variability  in  back/torso  discomfort  scores  may  have  been 
the  result  of  the  limited  number  of  back-intensive  operations  within  the  sample  group.  A  total  of 
20  questions  were  developed.  Eighteen  (90%)  were  tested,  two  had  high  variability.  Of  the  18 
tested,  16  (89%)  had  Kappa  values  greater  than  .40. 

The  agreement  rates  are  similar  to  those  obtained  by  Dickinson  et  al  [24]  who  reported 
agreement  rates  (not  chance  corrected)  of  .74  to  .94  for  yes/no  questions  regarding  discomfort 
frequency.  The  agreement  rates  (not  chance  corrected)  for  the  yes/no  questions  regarding  the 
occurrence  of  discomfort  ranged  from  .78  to  .90  in  this  pilot  test.  The  agreement  rates  for  the 
general  questions  regarding  discomfort  severity  and  pre-existing  conditions  were  similarly  high 
with  agreement  rates  between  .85  and  1 .00.  Results  are  presented  in  Table  3.7. 

The  test/re-test  reproducibility  for  the  Discomfort  Factors  section  of  the  Survey  is  similar  to  that 
of  other  discomfort  survey  tools  foxmd  in  the  literature.  This  suggests  that  the  modifications 
made  to  existing  questions  for  inclusion  in  the  Survey  did  not  negatively  impact  the 
reproducibility  of  the  questions. 

Table  3.7  Weighted  Kappa  Statistics  for  the  Discomfort  Factor  and  General  Questions 


Question 

Raw  % 
Agreement 

Weighted 

Kappa 

Comments 

46.  Shoulder/Neck 
Occurrence 

83% 

.646 

.411 

.881 

47.  S/N  Frequency 

75% 

.629 

.406 

.852 

48.  S/N  Severity 

68% 

.491 
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Table  3.7  Weighted  Kappa  Statistics  for  the 
Discomfort  Factor  and  General  Questions  (Contd.) 


Question 

Raw  % 
Agreement 

Weighted 
;  Kappa 

95% 

LCI 

95% 

UCI 

Comments 

49.  HandAVrist/Arm 
Occurrence 

85% 

.698 

.476 

.920 

50.  HAV/A 

Frequency 

65% 

.657 

.484 

.830 

51.  H/W/A  Severity 

63% 

Non-square 

52.  Back  Torso 
Occurrence 

78% 

.529 

.263 

.795 

53.  B/T  Frequency 

78% 

.371 

.145 

.598 

High  response  variability 

54.  B/T  Severity 

63% 

.340 

High  response  variability 

55.  Legs/Feet 
Occurrence 

90% 

.608 

56.  L/F  Frequency 

83% 

.587 

.886 

57.  L/F  Severity 

85% 

■ 

Non-square 

58.  Head/Eyes 
Occurrence 

85% 

.681 

.914 

59.  H/E  Frequency 

73% 

.579 

.345 

ma 

60.  H/E  Severity 

73% 

.524 

.276 

MM 

61.  Health  care  visits 

95% 

.724 

62.  Discomfort  not 
improving 

93% 

.754 

63.  Interference  with 
activities 

85% 

.601 

.316 

.887 

64.  Previous 
diagnosis 

88% 

.474 

.867 

65.  Contributing 
conditions 

100% 

1.00 

3.3.2.5  Work  Content  (Part  IIP 

During  the  reproducibility  testing  sessions  the  administrators  noticed  that  some  participants  were 
unclear  about  how  to  respond  to  tasks  which  they  did  not  perform.  While  the  original 
Administrator’s  Script  instructed  participants  to  “leave  blank”  tasks  that  they  “never”  performed, 
some  participants  continued  to  respond  to  all  tasks  (e.g.,  some  answered  “non-routine”  and 
several  answered  “seasonal”),  despite  the  instructions.  It  also  appears  that  some  participants 
selected  a  different  response  choice  for  these  tasks  in  the  re-test  administration.  The  resulting 
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weighted  Kappa  values  were  typically  in  the  poor  to  fair  range,  generally  between  .25  and  .45. 
The  results  are  presented  in  Table  3.8. 

A  “never”  category  was  added  to  the  Survey  after  the  completion  of  reproducibility  testing.  The 
agreement  on  the  revised  tool  is  expected  to  be  considerably  higher  as  a  result  of  this 
modification. 


Table  3.8  Weighted  Kappa  Statistics  for  Work  Content 


Question 

Raw  % 
Agreement 

Weighted 

Kappa 

95%  LCI 

95%  UCI 

66. 

28% 

.132 

.096 

0.360 

67. 

98% 

.952 

.861 

1.043 

68. 

75% 

.666 

.471 

.862 

69. 

28% 

.141 

-0.010 

.293 

70. 

60% 

.442 

.224 

.661 

71. 

58% 

.417 

.186 

.647 

72. 

50% 

.244 

.002 

.487 

73. 

68% 

.560 

.349 

.771 

74. 

88% 

75. 

28% 

.125 

-.071 

.321 

76. 

95% 

77. 

73% 

.513 

.275 

.751 

78. 

73% 

.618 

.419 

.817 

79. 

48% 

.290 

.061 

.519 

80. 

50% 

.338 

.098 

.578 

81. 

95% 

82. 

60% 

.476 

.245 

.709 

83. 

70% 

.370 

.085 

.655 

84. 

50% 

.294 

.047 

.542 

85. 

55% 

86. 

65% 

87. 

58% 

.447 

.225 

.670 

88. 

53% 

.374 

.144 

.603 

89. 

58% 

.327 

.068 

.587 
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Table  3.8  Weighted  Kappa  Statistics  for  Work  Content  (Contd.) 


Question 

Raw  % 
Agreement 

Weighted 

Kappa 

95%  LCI 

95%  UCI 

90. 

60% 

.389 

.136 

.642 

91. 

63% 

.384 

.125 

.643 

92. 

73% 

.438 

.156 

.719 

93. 

65% 

.494 

.261 

.726 

94. 

70% 

.516 

.265 

.767 

95. 

55% 

.323 

.064 

.581 

96. 

78% 

.701 

.508 

.895 

97. 

70% 

.413 

.131 

.695 

98. 

58% 

.318 

.057 

.579 

99. 

65% 

.476 

.225 

.727 

100. 

68% 

.485 

.221 

.749 

101. 

70% 

.412 

.121 

.703 

102. 

65% 

.509 

.262 

.756 

103. 

65% 

.337 

.049 

.625 

104. 

68% 

.530 

.284 

.776 

105. 

60% 

106. 

65% 

.446 

.183 

.709 

107. 

65% 

108. 

70% 

.525 

.269 

.781 

109. 

58% 

.326 

.054 

.599 

110. 

55% 

.450 

.218 

.681 

111. 

58% 

.295 

.012 

.578 

112. 

63% 

.402 

.127 

.677 

113. 

68% 

.507 

.250 

.764 

114. 

60% 

115. 

58% 

116. 

63% 

.456 

.200 

.711 

117. 

70% 

.400 

.106 

.694 

118. 

63% 

.384 

.106 

.662 
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Table  3.8  Weighted  Kappa  Statistics  for  Work  Content  (Contd.) 


Question 

Raw  % 
Agreement 

Weighted 

Kappa 

95%  UCI 

119. 

58% 

.340 

.074 

.605 

120. 

53% 

3.3.2.6  Shop  Level  Scores 

A  post  hoc  comparison  was  made  between  the  test  and  the  re-test  condition  for  the  priority  scores 
and  the  Survey  Priority  Rank  at  the  shop  level  using  a  Spearman  correlation.  This  comparison 
provides  preliminary  information  regarding  the  consistency  of  scores  at  the  shop  level.  The 
limited  number  of  shops  involved  (5  shops)  resulted  in  relatively  low  power  (estimated  power 
less  than  .30). 

Participants  who  were  not  present  for  both  administrations  were  removed  from  the  shop  scoring 
for  the  analysis.  The  power  was  weak  for  this  analysis  because  only  five  shops  were  represented. 
Furthermore,  the  opportunity  for  variance  was  high  because  two  of  the  shops  had  a  low  number 
of  participants.  The  Structural  Repair  Shop  had  five  participants  and  the  Survival  Equipment 
Repair  Shop  had  three  participants.  Since  the  Survey  scores  are  based  on  the  percent  of 
respondents,  the  scores  can  be  greatly  influenced  by  one  person’s  response  in  smaller  shops.  In 
spite  of  the  limitation,  the  preliminary  results  are  promising.  The  relative  risk  ranking  remained 
similar  between  both  Survey  administrations.  The  recommendations  regarding  EPRA  status  was 
unchanged  for  each  shop  across  the  two  administrations.  These  results  are  sufficiently  promising 
to  encourage  additional  testing  of  shop  level  reproducibility  using  a  sample  size  of  approximately 
25  shops.  The  shop  Survey  Priority  Rankings  were  determined  and  are  presented  in  Table  3.9. 

Table  3.9  Test/Re-Test  Shop  Priority  Rankings  Compared 


Shop 

Test  Rank 

Re-Test  Rank 

Dental  Lab 

7 

7 

Structural 

5 

7 

Survival  Equipment 

7 

5 

Falcon 

2 

2 

Bio/PH 

2 

1 

The  data  also  suggests  that  the  rank-order  of  body  areas  priority  scores  remained  consistent  from 
the  test  session  to  the  re-test  session  for  all  priority  scores  except  for  the  head/eyes.  In  spite  of 
the  power  limitations,  these  results  are  promising.  The  results  of  the  Spearman  correlation  on  the 
priority  scores  and  final  ranking  score  are  provided  in  Table  3.10. 
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Table  3.10  Spearman  Correlation  Between  Priority  and  Final  Ranking  Scores 


Body  Area 

Spearman 

Probability 

Shoulder/Neck 

1.00 

.000 

Hand/Wrist/Arm 

.82 

.086 

Back/Torso 

1.00 

.000 

Legs/Feet 

.65 

.237 

Head/Eyes 

-.02778 

.965 

Survey  Priority  Rank 

.73 

.161 

3.S.S  Validity  Testing 

The  correlation  between  the  overall  shop  rankings  determined  by  the  Ergonomist  and  the  Survey 
Priority  Rank  produced  by  the  Survey  demonstrated  a  statistically  significant  correlation  (p 
<.03),  although  the  correlation  obtained,  .39,  was  lower  than  the  estimated  .5  to  .7.  The 
ergonomist  determined  that  20  of  the  31  shops  could  be  designated  as  “problem/EPRA”  shops 
(e.g.,  high,  high/medium).  The  Survey  indicated  that  17  of  31  shops  could  be  designated  as 
EPRA  (e.g..  Priority  Rank  Score  >  or  =  5).  The  determination  of  EPRA  status  for  these  17 
shops,  however,  was  based  only  on  the  Survey  Priority  Rank  --  without  the  EWG  considering 
additional  information  on  each  shop.  For  example,  three  additional  shops  had  a  Priority  Rank  of 
4.  If  other  influencing  factors  were  present  (e.g.,  previous  reported  injuries),  these  same  three 
shops  could  reasonably  be  “upgraded”  to  EPRA  status.  Therefore,  the  rate  at  which  the 
Ergonomist  and  the  Survey  identified  “EPRA”  shops  is  comparable. 

The  criteria  correlation  of  .5  to  .7  was  a  broad  estimate  since  there  is  no  research  available  that 
has  attempted  to  demonstrate  this  type  of  correlation.  The  correlation  range  was  based  on  inter¬ 
rater  agreement  rates  from  previous  studies  [1],  [2],  [27].  The  original  estimate  was  also  based 
on  the  assumption  that  the  Survey  was  to  be  more  comprehensive;  where  risk  factors  would  be 
examined  and  rated  according  to  job  category  (e.g.,  separate  risk  factor  questions  for 
Administrative,  M/I,  Warehouse,  and  Assembly  work  areas).  Furthermore,  it  was  predicted  that 
the  Ergonomist’s  risk  rankings  would  have  a  higher  correlation  with  the  risk  factor  ratings  in  the 
Survey  than  with  the  overall  Survey  Priority  Rank,  since  the  Priority  Rank  eiIso  considered 
discomfort.  The  correlation  between  the  body  area  risk  rankings  of  the  Ergonomist  and  the 
Survey  ranged  from  -0.02  to  .50.  The  strongest  correlation’s  were  for  upper  body  risk  factors. 
The  Spearman  Rho  and  weighted  Kappa  value  for  each  body  area  comparison  are  presented  in 
Table  3.11. 
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Table  3.11  Spearman  Rho  and  Weighted  Kappa  Statistics  for  Each  Body  Area: 
Comparison  Between  Survey  and  Ergonomist  Expert  Results 


Body  Area 

Rho 

p.< 

95%  Cl 

Shoulder/Neck 

.46 

.01 

.25 

.01  to  .48 

Hand/Wrist/Arm 

.50 

.004 

.24 

.03  to  .44 

Back/Torso 

.03 

.89 

-.05 

-.25  to.  15 

Legs/Feet 

-0.02 

.91 

non-square 

Head/Eyes 

.30 

.11 

non-square 

The  agreement  trends  were  consistent  using  both  the  Spearman  Rho  and  weighted  Kappa. 
Agreement  on  upper  body  areas  was  higher  than  the  other  areas.  Particularly  interesting  was  the 
nature  of  disagreements,  the  level  at  which  the  Survey  rated  the  risk  compared  to  the  Ergonomist. 
With  the  exception  of  the  legs/feet  body  area,  the  Ergonomist  tended  to  rate  the  risk  within  the 
shop  at  a  higher  level  than  did  the  Survey.  The  frequency  with  which  each  method  rated  the  risk 
higher  is  presented  in  Table  3.12,  accompanied  by  the  agreement  frequency. 

Table  3.12  Agreement  Frequency  by  Body  Zone:  Comparison  Between  Survey  and 

Ergonomist  Expert  Results 


Body  Area 

Agreement 

Ranked 
Shop  Higher 

OVERALL  SHOP 

20 

7 

4  . 

Shoulder/Neck 

13 

14 

4 

HandAVrist/Arm 

12 

16 

3 

Back/Torso 

8 

17 

6 

Legs/Feet 

12 

3 

16 

Head/Eyes 

6 

5 

In  a  total  of  five  out  of  the  seven  cases  in  which  the  Ergonomist  ranked  the  shop  higher  risk  than 
did  the  Survey,  the  low  discomfort  rate  in  the  shop  was  a  primary  reason  for  the  lower  Survey 
ranks.  Two  factors  explain  the  difference.  First,  the  validation  process  required  that  the 
Ergonomist  be  kept  blind  to  discomfort  data.  Second,  in  the  Survey  scoring  process,  discomfort 
data  is  weighed  heavier  than  risk  factor  data.  Because  of  these  two  factors,  the  comparison  data 
provides  a  disagreement  rate  which  is  artificially  high.  Furthermore,  the  above  explanation 
indicates  that  the  agreement  regarding  ergonomic  risk  factors  in  the  job  (separate  from 
discomfort)  is  actually  greater  than  the  reported  agreement  between  the  Ergonomist  and  the 
Survey.  Table  3.13  presents  the  comparison  of  results  between  the  Ergonomist  and  the  Survey. 
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Table  3.13  EPRA  Classification  Rates:  Comparison  Between  Survey  and 

Ergonomist  Expert  Results 


Ergonomist  Expert 
EPRA 

Ergonomist  Expert 
Non-EPRA 

Survey  EPRA 

13 

4 

Survey  Non-EPRA 

7 

7 

The  Ergonomist’s  shop  classifications  as  EPRA  and  Non-EPRA  can  be  assumed  to  be  the  true 
classification,  in  spite  of  the  limitations  identified  below,  for  the  purposes  of  determining  how 
well  the  Survey  screening  tool  correctly  classifies  shops  by  risk  status.  The  PV  positive 
(predictive  value  of  a  positive  score)  of  the  Survey  was  calculated  as  76%.  This  indicates  that  an 
EWG  can  have  a  relatively  high  degree  of  certainty  (over  75%)  that  the  shops  they  are  targeting 
for  further  follow-up  (classified  as  EPRAs)  are  in  fact  shops  that  need  attention.  If  an  EWG 
wants  greater  certainty  that  the  shops  classified  as  EPRAs  by  the  Survey  are  indeed  shops  that 
need  attention,  the  cut-off  priority  score  could  be  raised  from  5  to  either  6  or  7.  This  increases 
the  PV  positive  to  85%.  It  should  be  noted,  however,  that  this  also  increases  the  likelihood  of 
misclassifying  a  shop  with  known  ergonomic  risk  factors  as  a  “non-EPRA”  shop. 

Thirteen  of  the  3 1  shops  used  in  this  comparison  study  did  not  have  the  desired  rate  of  employee 
participation  of  80%.  When  these  shops  with  lower  participation  rates  are  dropped  from  the 
comparison,  PV  positive  is  raised  to  80%  and  PV  negative  is  raised  to  75%  (from  .50).  Table 
3.14  reports  the  agreement  rates  for  shops  with  at  least  80%  response  rates.  The  small  sample 
size  (18  remaining  shops  instead  of  original  31)  suggests  that  these  results  need  to  be  interpreted 
cautiously.  Because  of  the  limitations  of  this  small  sample  size,  the  interpretation  of  results 
should  remain  on  the  findings  from  the  full  group  of  3 1  shops.  This  is  a  conservative  approach 
since  the  results  fi'om  the  smaller  group  suggest  even  better  tool  performance.  Table  3.15 
illustrates  the  effects  of  participation  rate  on  EPRA  agreement.  The  31  shops  are  listed  in 
descending  order  of  participation  rates,  -with  the  far  righthand  column  denoting  agreement 
between  the  ergonomist  and  the  Survey  regarding  EPRA  status.  The  established  target  of  an 
80%  response  rate  appears  justified  by  the  increasing  rate  of  disagreements  when  participation 
drops  below  80%.  These  results  clearly  indicate  the  importance  of  obtaining  a  high  response  rate 
and  that  caution  must  be  exercised  when  interpreting  Survey  results  from  shops  with  less  than  an 
80%  participation  rate.  Table  3.14  indicates  that  the  majority  of  the  improvement  in 
classification  agreement  occurred  in  the  reduction  of  false  negatives.  While  overall  agreement 
improved  from  65%  to  78%,  PV  negative  improved  firom  50%  to  75%.  These  findings  also 
suggest  that,  in  spite  of  some  methodological  differences  in  the  focus  of  the  Ergonomist  and  the 
Survey,  the  EPRA  classification  agreement  between  an  experienced  Ergonomist  and  the  Survey 
is  substantial.  Such  agreement  is  the  desired  intent  of  the  Survey  process. 
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Table  3.14  EPRA  Classification  Rates:  Comparison  Between  Survey  and  Ergonomist 
Results  Based  on  the  18  Shops  with  80%  or  Higher  Response  Rates 


EPRA 

Non-EPRA 

Survey  EPRA 

8 

2 

Survey  Non-EPRA 

2 

6 

Table  3.15  EPRA  Classification  Rates  and  Shop  Response  Rates: 
Comparison  Between  Survey  and  Ergonomist 


Base 

Organ 

Workplace 

Response 

Ergonomist 

RF/DS 

Agreement 

PAFB 

301  RQS 

Structural  Maintenance 

100% 

EPRA 

EPRA 

yes 

PAFB 

DECA/MSC 

Commissary-Meat 
Cutting  Room 

100% 

EPRA 

EPRA 

yes 

PAFB 

45  DS/SGD 

Dental  Lab 

100% 

EPRA 

EPRA 

yes 

PAFB 

45  SVS/SVRL 

Library 

100% 

EPRA 

EPRA 

yes 

PAFB 

301  RQS/MAF 

Hydraulics 

100% 

EPRA 

Non-EPRA 

PAFB 

45  MDG/SGOP 

Medical  Records 

100% 

EPRA 

EPRA 

yes 

PAFB 

45  SW/SESE 

Systems  Safety 

EPRA 

yes 

PAFB 

45  TRNS/LGTTS 

Packing  &  Crating 

100% 

EPRA 

EPRA 

yes 

PAFB 

45  MDG/SGOPA 

Appointment  Desk 

100% 

EPRA 

EPRA 

yes 

PAFB 

45  CES/CEOHVI 

Vertical  Construction 

100%  . 

Non-EPRA 

Non-EPRA 

yes 

PAFB 

45  CES/CEOIUF 

Liquid  Fuels 
Maintenance 

100% 

Non-EPRA 

Non-EPRA 

yes 

PAFB 

45  CS/SCMMG 

Radio  Maintenance 
Work  Center 

100% 

Non-EPRA 

Non-EPRA 

yes 

PAFB 

41  RQS/DOTL 

Life  Support 

100% 

Non-EPRA 

Non-EPRA 

yes 

PAFB 

DPS/DBO 

Reproduction  Shop 

EPRA 

CCAS 

6SWS 

Administrative 

Assistant 

100% 

EPRA 

Non-EPRA 

CCAS 

6SWS 

Entry  Controller 

Non-EPRA 

yes 

PAFB 

45  CS/SCM 

Cable/Telephone 

Maint. 

86% 

Non-EPRA 

Non-EPRA 

yes 

CCAS 

6SWS 

MWOC 

80% 

Non-EPRA 

PAFB 

DECA 

Commissary  Whse. 

78% 

EPRA 

yes 

PAFB 

45  CES/CEOHH 

Horizontal 

Construction 

76% 

EPRA 
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Base 

Organ 

Workplace 

Response 

Ergonomist 

RJ/DS 

Agreement 

PAFB 

45  CES  Zone  2 

Facility  Maint.  Zone  2 

75% 

Non-EPRA 

EPRA 

PAFB 

RAYTHEON 

Shipping  and 

Receiving 

73% 

EPRA 

EPRA 

yes 

PAFB 

45MDG 

Dental  Treatment 

72% 

,  EPRA 

EPRA 

yes 

PAFB 

DECA/SO/PAT 

Commissary 

71% 

EPRA 

EPPIA 

yes 

PAFB 

45  SW/XP 

Wing  Plans 

69% 

EPRA 

Non-EPRA 

PAFB 

741  MS/MAES 

Survival  Equipment 

67% 

EPRA 

EPRA 

yes 

CCAS 

6SWS 

Administrative 

Assistant 

66% 

EPRA 

Non-EPRA 

PAFB 

45  TRNS/LGTTF 

Air  Terminal 

56% 

EPRA 

Non-EPRA 

PAFB 

741  MS/MACA 

Aerospace  Ground 
Equipment 

56% 

Non-EPRA 

EPRA 

PAFB 

45  CES/CEH 

Housing  Office 

47% 

EPRA 

Non-EPRA 

CCAS 

6SWS 

Security  Controller 

40% 

Non-EPRA 

Non-EPRA 

yes 

The  results  indicated  that  a  survey  methodology  could  provide  similar  screening  information 
compared  to  the  method  of  having  an  experienced  Ergonomist  visit  individual  shops  and  assess 
the  ergonomics  hazard.  In  most  cases,  the  Ergonomist  and  the  Survey  agreed  on  the  decision  to 
recommend  status  as  an  EPRA  shop.  This  was  the  primary  consideration  in  testing  the  Survey 
tool,  and  the  results  indicate  that  the  tool  performs  its  intended  function. 

Although  the  Survey  performed  well,  the  contractor  investigated  the  potential  that  the 
performance  could  be  enhanced  through  changes  in  the  scoring  cut-offs.  The  impact  of  altering 
the  Job  Factor  (risk  factor)  exposure  percentages  for  obtaining  ratings  on  the  Survey  was 
investigated.  The  back/torso  area  was  used  as  the  basis  for  testing  the  change,  since  agreement 
was  the  lowest.  One  shop  was  dropped  from  this  step  due  to  missing  data  related  to  raw 
percentages.  A  scatter-plot  was  created  to  compare  Ergonomist  ratings  with  tally  percents. 
Based  on  the  tally  percent  distributions,  new  cut-offs  of  20%  for  Medium  and  40%  for  High  were 
tested.  Agreement  improved  to  14,  with  the  Ergonomist  rating  10  jobs  higher  and  the  Survey 
rating  six  jobs  higher.  Spearman  Rho  and  weighted  Kappa  values  were  both  increased  to  .12  and 
.08,  respectively.  However,  neither  of  these  tests  indicated  sufficient  agreement  to  justify 
concluding  significant  improvement.  Applying  the  20%  and  40%  cut-offs  to  the  shoulder/neck 
Survey  section  resulted  in  slight,  but  not  significant,  reduction  in  agreement.  There  is  no 
indication  that  altering  the  percent  cut-offs  in  the  Survey  for  determining  risk  exposure  would 
result  in  significant  improvements  in  the  agreement  between  the  Ergonomist  and  the  Survey. 

The  agreement  rates  may  have  been  higher  still  if  the  Ergonomist  had  not  been  intentionally  kept 
blind  to  all  aspects  of  survey  development.  The  results  can  be  summarized  by  considering  the 
advantages  and  disadvantages  of  keeping  the  Ergonomist  blind  to  the  Survey  and  Priority  Rank 
methodology.  The  obvious  advantage  was  that,  since  the  Ergonomist  remained  blind,  the  risk  of 
biasing  results  to  match  those  expected  by  the  Survey  were  minimized.  In  addition,  during  the 
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shop  visit,  the  Ergonomist  was  unable  to  “prep”  employees  for  how  to  interpret  or  answer  the 
Survey  questions.  Employees,  therefore,  were  able  to  complete  the  Survey  witliout  any 
predisposition.  The  disadvantages  and  possible  explanations  for  the  lower  than  predicted 
agreement  between  the  Ergonomist  and  the  Survey  are  listed  below: 

•  Difference  in  judgment  between  average  vs.  worst  exposures.  The  Survey  focused  on 
the  average  exposure  within  a  shop.  Since  the  Ergonomist  was  kept  blind  to  the  Survey 
methodology,  he  may  have  placed  more  focus  on  the  worst  of  the  exposures  observed 
Avithin  a  shop  during  the  visit  rather  than  an  average  of  all  exposures  -  and  rated  the  shop 
higher.  This,  in  fact,  was  the  case  particularly  for  the  back/torso  body  zone  and  for 
shops  whose  tasks  were  low  in  frequency  and  duration,  but  whose  biomechanical 
demands  (e.g.,  severity)  were  extreme.  When  these  types  of  tasks  were  observed  (and 
sometimes  demonstrated  as  “representative”),  the  Ergonomist  gave  a  higher  rank  than 
that  which  was  provided  by  the  Survey. 

•  Absence  of  information  on  discomfort.  Discomfort  information  could  not  be  provided 
to  the  Ergonomist  without  potentially  biasing  the  ranking.  Since  the  Ergonomist  had  no 
information  on  discomfort  and  since  the  Survey  Priority  Rank  is  weighted  more  heavily 
on  discomfort  than  risk  factor  exposure,  another  source  for  variation  is  introduced. 

•  Use  of  past  reported  incidents  as  a  primary  means  for  ranking  shops.  The 
Ergonomist  considered  risk  factor  exposure  to  the  same  body  zones  as  is  used  in  the 
Survey.  However,  he  also  used  evidence  of  past  incidents  (e.g.,  WMD)  to  establish  shop 
rank.  The  Survey  considers  the  use  of  past  incidents  in  interpreting  the  Survey  Priority 
Rank,  not  in  establishing  the  Survey  Priority  Rank. 

•  Low  response  rates  from  a  number  of  shops.  Thirteen  of  the  3 1  shops  had  five  or  less 
respondents,  with  several  shops  having  only  one  or  two  respondents.  Since  the  Survey 
results  are  based  on  percentages  of  people  exposed  to  risk  factors  and  reporting 
discomfort,  results  based  on  small  numbers  of  respondents  are  subject  to  large 
fluctuations.  For  example,  a  single  response  from  one  person  could  create  the  difference 
between  a  low  and  high  risk  shop. 

•  Simplification  of  the  Survey  design.  Initial  drafts  of  the  Survey  were  simplified  to  meet 
the  time-for-completion  objectives  established  by  the  Air  Force.  Two  significant 
simplifications  which  would  be  cause  for  re-evaluating  (e.g.,  lowering)  the  original 
agreement  estimate  (.5  to  .7)  included: 

-  eliminated  assessment  of  risk  factor  exposure  by  work  area  type  (e.g..  Administrative, 
M/I,  Warehouse,  and  Assembly)  in  favor  of  one  general  Job  Factors  section;  and 
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-  eliminate  the  “weighting”  of  several  risk  factors  in  scoring  in  favor  of  establishing  the 
“weight”  of  each  risk  factor  as  equal  in  order  to  simplify  and  increase  the  speed  of  the 
scoring  process. 

The  Survey  may  function  best  when  all  of  its  components  are  used  when  reaching  a  decision  on 
EPRA  status.  In  addition  to  the  Priority  Rank,  the  following  must  be  used  when  making  the 
EPRA  determination: 

•  influence  of  organizational  factors; 

•  influence  of  employee  perception  of  physical  effort; 

•  influence  of  health  or  other  conditions  that  may  impact  reported  discomfort;  and 

•  history  of  past  reported  incidents. 

This  reinforces  the  decision  to  place  the  final  determination  of  EPRA  status  and  strategy  for 
intervention  into  the  hands  of  the  EWG  supported  by  information  provided  by  the  Survey. 

3.4  Discussion 

The  test/retest  reproducibility  of  the  individual  questions  was  evaluated  to  determine  if  the 
modifications  to  the  questions  had  altered  their  reproducibility  from  earlier  studies.  The  Kappa 
values  obtained  on  individual  questions  were  generally  equivalent  indicating  that  the  alterations 
to  questions  had  minimal  impact  ([24],  [4]).  The  agreement  on  the  scale  scores  is  also 
comparable  to  previous  findings.  The  Kappa  values  ranging  up  to  .68  for  the  body  part  scale 
scores  are  similar  to  the  Spearman  Rho  scores  of  .69  to  .82  for  the  repetition,  force,  and  whole 
body  activity  scales  reported  by  Cole  [1]. 

This  suggests  that  the  modifications  made  to  the  questions  and  answers  categories  had  a  minimal 
impact  on  reproducibility.  In  addition,  the  new  questions  which  were  created  specifically  for  the 
Survey  (based  on  risk  factors  reported  in  the  literature)  had  reproducibility  rates  similar  to 
questions  fovmd  in  existing  surveys,  as  did  questions  which  were  adapted  from  questions  foimd 
in  existing  surveys. 

The  reproducibility  and  validity  of  the  Survey  appears  stronger  for  upper  extremity  concerns 
(shoulder/neck  and  hands/wrists/arms)  than  for  the  other  body  areas.  This  needs  to  be  taken  into 
consideration  when  the  EWG  makes  the  final  EPRA  determination,  especially  when  dealing  with 
“borderline”  jobs.  For  example,  if  the  hand/wrist/arm  score  determined  an  Survey  Priority  Rank 
of  5,  the  EWG  should  be  discouraged  from  excluding  the  shop  from  EPRA  status  based  on  the 
other  considerations.  If,  on  the  other  hand,  the  legs/feet  score  determined  the  overall  Priority 
Rank  of  4,  the  EWG  is  encouraged  to  carefully  review  the  other  considerations  before  reaching  a 
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final  decision  on  EPRA  status.  Depending  on  the  other  considerations,  that  shop  might  be 
upgraded  to  EPRA  status. 

3.5  Conclusions 

The  Survey  performs  effectively  and  efficiently  as  an  active  surveillance/screening  tool.  The 
strengths  of  the  Survey  are  listed  below: 

•  The  Survey  is  quick  and  easy  to  administer.  The  Survey  can  be  administered  to  and 
completed  by  a  group  (unlimited  size)  of  assembled  employees  in  approximately  45 
minutes. 

•  The  Survey  is  quick  and  easy  to  score.  In  the  most  recent  trials,  scoring  for  a  shop  of  25 
employees  was  completed  by  Public  Health  in  less  than  2  hours. 

•  Parts  II  (Work  Content)  and  Parts  IV  (Process  Improvement  Opportunities)  enable 
employees  to  categorize  their  routine  types  of  work  processes,  activities,  and  tasks, 
according  to  standardized  categories.  Part  IV  specifically  enables  employees  to  comment 
directly  on  the  tasks,  tools,  equipment,  materials,  etc.  that  they  feel  most  relates  to  their 
perceived  exposure  to  ergonomic  risk  factors  or  personal  experience  with  discomfort  or 
fatigue.  Information  from  both  Parts  can  be  used  by  Public  Health  and  the  EWG  to 
design  efficient  intervention  strategies  as  well  as  communicate  requests  for  follow-up  by 
Bioenvironmental  Engineering  Services. 

•  Completion  of  the  Survey  within  a  shop  provides  an  Ergonomic  Shop  Priority  Rank 
which  enables  Public  Health  and  the  EWG  to  make  an  initial  determination  of  EPRA 
status.  The  Survey  Priority  Rank,  in  combination  with  other  considerations  such  as  past 
reported  WMDs,  organizational  factors,  perceived  physical  effort,  etc.,  enables  the  EWG 
to  make  a  final  determination  of  EPRA  status  based  on  a  thoughtful  interpretation  of  the 
common  indications  of  the  data.  The  methodology  recognizes  the  value  in  achieving  a 
balance  between  the  Survey  results,  professional  expertise,  and  shop  experience. 

•  The  numerical  based  Survey  results  can  be  used  to  prioritize  EPRA-classified  work  areas 
for  “task  specific”  analyses  and/or  problem-solving  work.  The  Survey  Priority  Rank  can 
be  used  to  establish  an  initial  priority  list  (e.g..  Priority  1  -  Shop  A,  Survey  Priority  Rank 
8;  Priority  2  -  Shop  B;  Survey  Priority  Rank  7;  and  so  on).  The  Work  Content  (Part  III) 
and  Process  Improvement  Opportunities  (Part  IV)  sections  provide  information  on  the 
processes,  tasks,  equipment,  etc.  that  may  be  the  targets  of  initial  action  for  follow-up. 

•  Results  of  the  Survey  provide  an  indication  of  and  the  relative  importance  of  ergonomic, 
psychosocial,  and  individual  factors  that  may  be  present  in  the  work  area.  Ergonomic 
factors  (e.g.,  job  factors,  discomfort  factors)  are  of  primary  importance  in  determining  the 
Survey  Priority  Rank  of  the  shop.  Psychosocial  factors  and  their  potential  impact  on  the 
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ergonomic  factors  can  be  considered  by  reviewing  the  Organizational  Ratings.  For 
example,  a  rating  of  High  in  the  Organizational  Factors  section  indicates  that  many 
people  in  the  shop  may  experience  a  high  level  of  job  stress.  High  levels  of  job  stress 
can  decrease  job  performance  and  increase  the  experience  of  pain  and  discomfort.  If  the 
Organizational  Rating  is  High,  it  suggests  that  a  follow-up  job  stress  evaluation  may  be 
used  as  follow-up.  Individual  factors  and  their  potential  impact  on  ergonomic  factors  can 
be  considered  by  reviewing  the  Contributing  Factors  Score.  This  percentage  provides 
insight  into  interpreting  the  Discomfort  Rating.  For  example,  if  the  Contributing  Factors 
score  is  above  20%,  the  Discomfort  Rating  could  have  been  impacted  by  a  high 
percentage  of  employees  with  conditions  that  increase  the  prevalence  of  WMDs. 

•  Data  from  the  Survey  allows  calculation  of  employee-reported  discomfort  prevalence 
rates.  Information  contained  in  the  Discomfort  Factors  section  enables  Public  Health  to 
calculate,  by  body  zone  (e.g.,  shoulder/neck,  hands/arms/wrists,  back/torso,  legs/feet,  and 
head/eyes),  the  percentage  of  employees  within  a  shop  who  are  experiencing  or  who  have 
experienced  discomfort  in  the  year  preceding  their  completion  of  the  Survey.  This 
information  may  also  be  used  by  Public  Health  to  gain  insight  into  the  effectiveness  of 
the  Air  Force  injury  and  illness  reporting  system  and  determine  whether  or  not  it  is  likely 
that  employees  are  under-reporting  their  musculoskeletal  discomfort  or  symptoms  of 
WMDs. 

In  addition,  the  Survey  has  similar  or  better  reproducibility  than  other  ergonomic  screening  tools 
reported  in  the  literature.  The  Survey  performs  best  in  shops  with  six  or  more  employees  and 
results  are  most  reliable  when  at  least  80%  employee  participation  is  obtained.  More 
importantly,  the  Survey  Methodology  provides  the  Air  Force  with  a  tool  that  is  unique  to  the 
field  of  ergonomics.  It  is  the  first  tool  for  which  reproducibility  has  been  reported  to  allow  for 
the  following:  (1)  enables  a  massive  organization  to  systematically  and  quickly,  with  a  minimum 
of  resources,  assess  employee  exposure  to  ergonomic  factors  in  all  types  of  work  environments; 
(2)  results  (Survey  Priority  Rank)  can  be  used  to  establish  overall  priorities  for  further 
investigation  at  the  shop  level;  (3)  results  (Work  Content  and  Process  Improvement 
Opportunities)  can  be  used  to  establish  a  plan  for  specific  follow-up  within  the  higher  priority 
shops;  and  (4)  can  be  used  to  measure  the  potential  impact  of  problem-solving  efforts  that  have 
been  completed  within  a  shop  and  for  all  shops  throughout  the  organization. 

Finally,  the  Survey  Methodology  provides  data  necessary  to  enable  the  Air  Force  to  maximize 
the  value  of  the  professional  expertise  and  experience  of  Public  Health  and  members  of  the 
EWG.  These  two  entities  are  charged  with  the  final  determination  of  EPRA  status  and  the  design 
of  an  intervention  strategy  to  prevent  WMDs  among  Air  Force  personnel. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification 


1  Question  -  1: 

I  work  with  my  hands  at  or  above  chest  level. 

Original  Question: 

Is  an  elbow  used  at  or  above  mid-torso  level? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  physical  landmark. 

Risk  Factor: 

Non-neutral  position  of  the  shoulder,  static  fatigue. 

Reference 

Bjelle,  A.,  Hagberg,  M.  and  Michaelsson,  G.  (1979).  Clinical 
and  ergonomic  factors  in  prolonged  shoulder  pain  among 
industrial  workers.  Scand.  J.  Work  Environ,  and  Health. 

(Vol.  5,  pp.  205-210). 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  International  Journal  of 
Industrial  Ergonomics.  (Vol.  ll,pp.,  pp.  51-65). 

Potential  WMD 

Shoulder  Bursitis,  Thoracic  Outlet  Syndrome,  Rotator  Cuff 
Tendonitis,  and  Upper  Back  Disorders. 

Question  -  2: 

To  get  to  or  to  do  my  work,  I  must  lay  on  my  back  or  side  and 
work  with  my  arms  up. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  demands  of  Maintenance  and 
Inspection  work. 

Risk  Factor  Basis: 

Non-neutral  position  of  the  shoulder,  static  fatigue. 

Reference 

Bateman,  J.E.  (1983).  Neurologic  painful  conditions  affecting 
the  shoulder.  Clin.  Orthop.  Rel.  Res.  (Vol.  173.,  pp.  44-54). 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue.  Definition  and 
measurement  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Potential  WMD: 

Shoulder  Bursitis,  Thoracic  Outlet  Syndrome,  Rotator  Cuff 
Tendonitis,  and  Upper  Back  Disorders. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification  (Contd.) 


Question  -  3 

I  must  hold  or  cany  materials  (or  large  stacks  of  files)  during  the 
course  of  my  work. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  work  requirements. 

Risk  Factor  Basis: 

Non-neutral  position  of  the  shoulder,  static  fatigue. 

Reference 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue:  Definition  and 
measurement  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Mital,  A.,  Nicholson,  A.S.,  and  Ayoub,  M.M.  (1993/  A  Guide 
to  Manual  Materials  Handling.  London,  England:  Taylor  & 
Francis. 

Potential  WMD: 

Shoulder  Bursitis,  Thoracic  Outlet  Syndrome,  Rotator  Cuff 
Tendonitis,  and  Upper  Back  Disorders. 

Question  -  4: 

I  force  or  yank  components  or  work  objects  in  order  to  complete 
a  task. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  demands  of  Maintenance  and 
Inspection  work. 

Risk  Factor  Basis: 

Non-neutral  position  of  the  shoulder,  high  speed  arm  motions. 

Reference 

Putz- Anderson,  V.  (1992).  Cumulative  trauma  disorders:  A 
manual  for  musculoskeletal  diseases  of  the  upper  limb. 
London,  England:  Taylor  &  Francis. 

Potential  WMD: 

Shoulder  Bursitis,  Thoracic  Outlet  Syndrome,  Rotator  Cuff 
Tendonitis,  Medial/Lateral  Epicondylitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification  (Contd.) 


Question  -  5: 

I  reach  or  hold  my  arms  in  front  of  or  behind  my  body  (e.g., 
using  a  keyboard,  filing,  handling  parts,  performing  inspection 
tasks,  pushing  or  pulling  carts,  etc.). 

Original  Question: 

Is  repeated  or  sustained  work  performed  when  one  arm  reaches 
forward  or  to  the  side  without  support? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor  (e.g.,  “hold”  replaces  “sustained 
work”)  and  addition  of  work  situation  examples  to  provide 
context. 

Risk  Factor  Basis: 

Non-neutral  position  of  the  shoulder,  static  fatigue. 

Reference 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue:  Definition  and 
measurement  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Corlett,  E.N.  (1983).  Analysis  and  evaluation  of  working 
postures.  In  T.O.  Kvalseth  (Ed.),  Ergonomics  of 

Workstation  Design,  (pp.  12-15).  London:  Butterworths. 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scandinavian  Journal  of 
Rehabilitative  Medicine.  (Vol.  26,  pp.  1-21). 

Nichols,  H.M.  (1967).  Anatomic  structures  of  the  thoracic 
outlet.  Clin.  Orthop.  Rel.  Res.  (Vol.  51,  pp.  17-25). 

Potential  WMD: 

Shoulder  Bursitis,  Thoracic  Outlet  Syndrome,  Rotator  Cuff 
Tendonitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification  (Contd.) 


Question  -  6: 

My  neck  is  tipped  forward  or  backward  when  I  work. 

Original  Question: 

Does  your  work  involve  that  you  hold  your  head  bent  forward? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Improve  clarity  of  risk  factor,  add  neck  bent  backward  as  another 
expectedly  common  and  stressful  work  position  for  the  neck. 

Risk  Factor  Basis: 

Non-neutral  position  of  the  neck,  static  fatigue. 

Reference 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue:  Definition  and 
measurement.  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Hagberg,  M.  (1984).  Occupational  musculoskeletal  stress  and 
disorders  of  the  neck  and  shoulder:  a  review  of  possible 
pathophysiology.  Int.  Arch.  Occup.  Environ.  Health.  (Vol. 

53,  pp.  269-278). 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  Int.  J.  Ind.  Ergon.  (Vol. 
ll,pp.  51-61). 

Van  Wely,  P.  (1970).  Design  and  Disease.  Appl.  Ergon.  (Vol., 
No.  5,  pp.  262-269). 

Wiktorin,  C.,  et  al.  (1993).  Validity  of  self— reported  exposures 
to  work  postures  and  manual  materials  handling.  Scand.  J. 
Work  Environ  Health,  (Vol.  19,  pp.  208-214). 

Potential  WMD: 

Disc  Degeneration  in  Cervical  Spine,  Tendonitis. 

Question  -  7: 

I  cradle  a  phone  or  other  device  between  my  neck  and  shoulder. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  work  situations  found  in 
administrative  and  M&I  tasks. 

Risk  Factor: 

Non-neutral  position  of  the  shoulder/neck,  neurovascular 
compression,  static  fatigue. 

Reference 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue:  Definition  and 
measurement.  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Dale,  W.A.  (1982).  Thoracic  outlet  compression  syndrome. 

Arch.  Surg.  (Vol.  117,  pp.  1437-1445). 

Tyson,  R.R.,  and  Kaplan,  G.F.  (1975).  Modem  concepts  of 
diagnosis  and  treatment  of  the  thoracic  outlet  syndrome. 
Orthop.  Clinics  of  North  America  (Vol.  6,  pp.  507-519). 

Potential  WMD 

Thoracic  Outlet  Syndrome,  Rotator  Cuff  Tendonitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification  (Contd.) 


Question  -  8: 

My  wrists  are  bent  (up,  down,  to  the  thumb  or  little  finger  side) 
while  I  work. 

Original  Question: 

Can  the  job  be  done  without  bending  the  wrist? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  direct  evaluation  of  required  wrist 
posture  rather  than  evaluation  of  task  and  possibility  of  change, 
example  of  bent  wrist  postures. 

Risk  Factor: 

Non-neutral  hand/wrist  positions. 

Reference 

Lifshitz,  Y.,  and  Armstrong,  T.  (1986).  A  Design  Checklist  for 
Control  and  Prediction  of  Cumulative  Trauma  Disorder  in 
Intensive  Manual  Jobs.  In  Proceedings  of  the  Human  Factors 
Society  30th  Annual  Meeting,  (pp.  945-950). 

Potential  WMD 

Hand/wrist  disorders:  Tendonitis,  Carpal  Tunnel  Syndrome. 

Question  -  9: 

I  apply  pressure  or  hold  an  item/material/tool  (e.g.,  screw  driver, 
spray  gun,  mouse,  etc.)  in  my  hand  for  longer  than  10  seconds  at  a 
time. 

Original  Question: 

Is  the  tool  continually  held  in  the  hand? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Applicability  of  question  expanded  to  all  tasks  that  may  require 
static  work  in  the  hands  rather  than  just  those  tasks  which  involve 
tool  use,  example  work  situations  added  to  provide  context. 

Risk  Factor: 

Prolonged  force  application. 

Reference 

Reynolds  et  al  (1994).  A  field  methodology  for  the  control  of 
musculoskeletal  injuries.  Applied  Ergonomics.  (Vol.  25,  No. 
l,pp.  3-16). 

Potential  WMD 

Tendonitis. 

Question  -  10: 

My  work  requires  me  to  use  my  hands  in  a  way  that  is  similar  to 
wringing  out  clothes. 

Original  Question: 

Can  the  job  be  done  without  “clothes  wringing”  motion? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  direct  evaluation  of  motion  rather  than 
evaluation  of  task  and  possibility  of  change. 

Risk  Factor: 

Non-neutral  wrist/arm/elbow  positions. 

Reference 

Lifshitz,  Y.,  and  Armstrong,  T.  (1986).  A  Design  Checklist  for 
Control  and  Prediction  of  Cumulative  Trauma  Disorder  in 
Intensive  Manual  Jobs.  In  Proceedings  of  the  Human  Factors 
Society  30th  Annual  Meeting  (pp.  945-950). 

Potential  WMD 

Tendonitis,  Carpal  Tunnel  Syndrome,  Medial/Lateral 

Epicondylitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
and  References,  and  Rationale  for  Question  Modification  (Contd.) 


Question  - 11: 

I  perform  a  series  of  repetitive  tasks  or  movements  during  the 
normal  course  of  my  work  (e.g.,  using  a  keyboard,  tightening 
fasteners,  cutting  meat,  etc.). 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  introduce  the  “repetition”  factor,  work 
situation  examples  provided  for  context. 

Risk  Factor: 

Frequency  of  similar  motions. 

Reference 

Kilbom,  A.  (1994).  Int.  J  Ind.  Ergon.  (Vol.  14,  pp.  59-86). 

Kuorinka,  I.,  and  Koshinen,  P.  (1979).  Occupational  rheumatic 
diseases  and  upper  limb  strain  in  manual  jobs  in  a  light 
mechanical  industry.  Scand.  J.  Work  Environ.  Health  (V ol. 

5,  No.  3,pp.  39-47). 

Potential  WMD 

Hand/wrist/shoulder/elbow  disorders:  Carpal  Tunnel  Syndrome, 
Tendonitis,  Epicondylitis,  ganglion  cysts. 

Question  - 12: 

The  work  surface  (e.g.,  desk,  bench,  etc.)  or  tool(s)  that  I  use 
presses  into  my  palm(s),  wrist(s),  or  against  the  sides  of  my 
fingers  leaving  red  marks  on  or  beneath  the  skin. 

Original  Question: 

Do  the  hands/wrists/arms  come  in  contact  with  any  sharp,  or 
non-roxmded  edges  on  the  table/machinery? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

First  person  verbiage,  work  situation  examples  added  to  provide 
context  (especially  for  administrative  work),  “sharp”  term 
discarded  to  keep  focus  on  instances  where  the  work  surface  puts 
pressure  on  the  body  region. 

Risk  Factor: 

External  trauma/ischemia. 

Reference 

Reynolds,  et  al.  (1994).  A  field  methodology  for  the  control  of 
musculoskeletal  injuries.  Applied  Ergonomics.  (Vol.  25,  No. 
l,pp.  3-16). 

Putz- Anderson,  V.  (1992).  Cumulative  trauma  disorders:  A 
manual  for  musculoskeletal  diseases  of  the  upper  limb. 
London,  England:  Taylor  &  Francis. 

Potential  WMD 

Neural  entrapment. 
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Question  - 13: 

I  use  my  hand/palm  like  a  hammer  to  do  certain  aspects  of  my 
work. 

Original  Question: 

Is  the  palm  or  base  of  the  hand  used  as  a  striking  tool  (like  a 
hammer)? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor. 

Risk  Factor: 

High  force  projection  and  non-neutral  hand/wrist  positions. 

Reference 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  Int.  J.  Ind.  Ergon.  (Vol. 

11,  pp.  51-61). 

Putz- Anderson,  V.  (1992).  Cumulative  trauma  disorders:  A 
manual  for  musculoskeletal  diseases  of  the  upper  limb. 

London,  England:  Taylor  &  Francis. 

Potential  WMD  Carpal  Tunnel  Syndrome,  Tendonitis. 

I  Question  -  14: 

My  hands  and  fingers  are  cold  when  I  work. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  the  effect  of  temperature 
extremes,  especially  cold,  on  the  hand/wrist/arm. 

Risk  Factor: 

Cold  temperature. 

Reference 

Heus,  R.,  Daanen,  H.A.M.,  and  Haventh,  G.  (1995). 

Physiological  criteria  for  functioning  of  hands  in  the  cold. 

Appl.  Ergon.  (Vol.  26,  No.  1,  pp.  5-13). 

Holmer,  I.  (1994)  Cold  Stress  -  Part  I:  Guide  for  the 

Practitioner.  Int.  J.  Ind.  Ergon.  (Vol.  14,  pp.  139-149). 

Potential  WMD 

Carpal  Tunnel  Syndrome,  HAVS. 
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Question  - 15: 

I  work  at  a  fast  pace  to  keep  up  with  a  machine  production  quota 
or  performance  incentive. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

Added  new  question  to  reflect  insufficient  recovery  time  risk 
factor. 

Risk  Factor: 

Incentive  based  production,  i.e.,  lack  of  time  for  rest/repair. 

Reference 

Feldman,  R.G.,  Goldman,  R.,  and  Keyserling,  W.M.  (1983). 
Peripheral  nerve  entrapment  syndromes  and  ergonomic 
factors.  Am.  J.  Ind.  Med.  (Vol.  4,  pp.  661-681). 

Ohara,  H.,  Aoyama,  H.,  Itani,  T.,  Nakagiri,  S.,  and  Wake,  K. 
(1976).  Occupational  health  hazards  resulting  jfrom  elevated 
work  rate  situations.  J.  Human  Ergon.  (Vol.  5,  pp.  173-182). 

Silverstein.  B.A.,  Fine,  L.J.,  and  Armstrong,  T.J.  (1987). 
Occupational  factors  and  Carpal  Tunnel  Syndrome.  Am.  J. 

Ind.  Med.  (Vol.  11,  pp.  343-358). 

Silverstein,  B.A.,  Fine,  L.J.,  and  Armstrong,  T.J.  (1986).  Hand/ 
wrist  cumulative  trauma  disorders  in  industry.  Br.  J.  of  Ind. 
Med.  (Vol.  43,  pp.  779-784). 

Smith,  M.J.,  Carayon,  P.,  Sanders,  K.J.,  Lim,  S.Y.,  and 

LeGrande,  D.  (1992).  Employee  stress  and  health 
complaints  in  jobs  with  and  without  electronic  performance 
monitoring. Ergon.  (Vol.  23,  No.  l,pp.  17-27). 

Potential  WMD 

Hand/wrist/elbow  disorders:  De  Quervain’s,  Tendonitis, 
ganglion  cyst.  Carpal  Tuimel  Syndrome,  Medial/Lateral 
Epicondylitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
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Question  -  16: 

The  tool(s)  that  I  use  vibrate  and/or  jerk  in  my  hand(s)  and 
aim(s). 

Original  Question: 

Does  the  tool  or  object  jerk  the  hand? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Combined  vibration  and  “torque”  (jerk)  risk  factors  to  minimize 
number  of  questions. 

Risk  Factor: 

Vibration  and  the  application  of  excessive  forces. 

Reference 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  Int.  J.  Ind.  Ergon.  (Vol. 
ll,pp.  51-61). 

Kihlberg,  S.  (1995).  Biod3mamic  response  of  the  hand-arm 
system  to  vibration  from  an  impact  hammer  and  grinder.  Int. 

J.  Ind.  Ergon.  (Vol.  16,  pp.  1-8). 

Potential  WMD 

Hand/wrist/elbow  disorders:  HAVS,  Carpal  Tunnel  Syndrome, 
Medial/Lateral  Epicondylitis. 

Question  -  17: 

My  work  requires  that  I  repeatedly  throw  or  toss  items. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  demands  of  Air  Force  work. 

Risk  Factor: 

High  speed  arm  motions. 

Reference 

Delisie,  A.,  Gagnon,  M.  (1995).  Segmental  dynamic  analysis 
when  throwing  loads.  Int.  J.  Ind.  Ergon.  (Vol.  16,  pp.  9-21). 

Potential  WMD 

Shoulder/arm/elbow  disorders:  Medial/Lateral  Epicondylitis, 
Rotator  Cuff  Tendonitis. 
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Question  - 18: 

My  work  requires  me  to  twist  my  forearms,  such  as  turning  a 
screwdriver. 

Original  Question: 

Is  repeated  work,  with  forearms  and  hand,  performed  with 
twisting  movements? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  increased  emphasis  on  the  twisting 
motion,  example  provided  for  context. 

Risk  Factor: 

Non-neutral  wrist/forearms  and  stressful  wrist  motions. 

Reference 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scandinavian  Journal  of 
Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

Armstrong,  T.,  Werner,  R.,  Waring,  W.,  and  Foulke,  J.  (1986). 
Intra-Carpal  Canal  Pressure  in  Selected  Hand  Tasks.  The 
University  of  Michigan. 

Chaffin,  D.B.  (1973).  Localized  muscle  fatigue:  Definition  and 
measurement.  J.  Occup.  Med.  (Vol.  15,  pp.  346-354). 

Putz- Anderson,  V.  (1992).  Cumulative  trauma  disorders:  A 
manual  for  musculoskeletal  diseases  of  the  upper  limb. 
London,  England:  Taylor  &  Francis. 

Potential  WMD 

Hand/wrist  disorders:  Carpal  Tunnel  Syndrome,  Tendonitis,  De 
Quervain’s  Syndrome.  Shoulder/elbow  disorders: 

Medial/Lateral  Epicondylitis. 
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and  References,  and  Rationale  for  Question  Modification  (Contd.) 


1  Question  - 19: 

I  wear  gloves  that  are  bulky,  or  reduce  my  ability  to  grip. 

Original  Question: 

Do  the  gloves  hinder  gripping? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor  (e.g.,  “reduce  my  ability”  replaces 
“hinder,”  question  reworded  to  reflect  that  some  employees  do 
not  wear  gloves. 

Risk  Factor: 

Increased  grip  force. 

Reference 

Batra,  S.,  Wang,  M.J.,  and  Bishu,  R.R.  (1994).  Glove  attributes: 
Can  they  predict  performance?  Int.  J.  Ind.  Ergon.  (Vol.  14, 
pp.  201-209). 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  Int.  J.  Ind.  Ergon  (Vol. 
ll,pp.  51-61). 

Nelson,  J.B.,  and  Mital  A.  (1995).  An  Ergonomical  Evaluation 
of  the  Primary  Hand  Flexibility  and  Capability  Changes  with 
Increases  in  Examination/Surgical  Glove  Thickness. 
Ergonomics  QJ o\.  38,  No.  4). 

Bishu,  R.R.,  and  Klute,  G.  (1995).  The  effects  of  external 
vehicular  activity  (EVA)  gloves  on  human  performance,  Int. 

J.  Ind.  Ergon.  (Vol.  16,  pp.  165-174). 

Potential  WMD 

Hand/wrist  disorders:  Carpal  Tunnel  Syndrome,  Tendonitis,  De 
Quervain’s  Syndrome.  Shoulder/elbow  disorders: 

Medial/Lateral  Epicondylitis. 

Question  -  20: 

I  apply  pressure  with  my  hands  similar  to  the  way  people  use 
their  hands  to  open  a  new  bottle  of  soda. 

Original  Question: 

Estimate  the  average  amormt  of  time  per  day  that  requires  as 
much  force  as:  unscrewing  a  bottle  cap  on  a  new  bottle  or 
container  of  pop  (a  bottle  or  container  that  has  never  been 
opened). 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Simplification  of  question,  used  term  “apply  pressure”  instead  of 
“force”  as  a  more  direct  statement  of  the  risk  factor. 

Risk  Factor: 

High  force  pinch  grips. 

Reference 

Cole,  L.L.  (1995).  Construction  and  Validation  of  a 
Musculoskeletal  Risk  Questionnaire.  Dissertation. 

Potential  WMD 

Hand/wrist  disorders:  Carpal  Tunnel  Syndrome,  Tendonitis,  De 
Quervain’s  Syndrome. 
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Question  -  21: 

I  grip  work  objects  or  tools  as  if  I  am  gripping  tightly  onto  a 
pencil. 

Original  Question: 

Is  a  pinch  grip  used? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

More  direct  statement  of  the  risk  factor,  example  of  pinch  grip 
provided  for  those  individuals  who  have  never  heard  the  term, 
added  a  “force”  component  to  the  question  to  distinguish  those 
individuals  who  may  be  applying  significant  finger  tip  force 
(rather  than  just  using  the  finger  tips). 

Risk  Factor: 

High  force  pinch  grips. 

Reference 

Dempsey,  P.G.,  and  Ayoub,  M.M.  (1996).  The  influence  of 
gender,  grasp  type,  pinch  width  and  wrist  position  on 
sustained  pinch  strength.  Ind.  J.  Ind.  Ergon.  (Vol.  17,  pp. 
259-273). 

Keyserling,  W.M.,  Brouwer,  M.,  and  Silverstein,  B.A.  (1993). 

The  effectiveness  of  a  joint  labor-management  program  in 
controlling  awkward  postures  of  the  trunk,  neck  and 
shoulders:  Results  of  a  field  study.  Int.  J.  Ind.  Ergon  (Vol. 
ll,pp.  51-61). 

Silverstein.  B.A.,  Fine,  L.J.,  and  Armstrong,  T.J.  (1987). 

Occupational  factors  and  Carpal  Tunnel  Syndrome.  Am.  J. 

Ind.  Med.  (Vol.  11,  pp.  343-358). 

Potential  WMD 

Hand/wrist  disorders:  Carpal  Tunnel  Syndrome,  Tendonitis,  De 
Quervain’s  Syndrome. 
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Question  -  22: 

When  I  lift,  move  components,  or  do  other  aspects  of  my  work, 
my  hands  are  lower  than  my  knees. 

Original  Question: 

Are  loads  lifted  manually?  Notice  factors  of  importance  as 
handling  below  knee  height. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  expanded  applicability  of  risk  factor 
(stressful  posture)  to  work  which  involves  handling  activities  in 
addition  to  lifting. 

Risk  Factor: 

Asymmetrical  lifting,  twisting. 

Reference 

Reynolds,  J.L.,  Drury,  C.G.,  and  Broderick,  R.L.  (1994).  A  field 
methodology  for  the  control  of  musculoskeletal  injuries. 

Appl.  Ergon.  (Vol.  25,  No.  1,  pp.  3-16). 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scandinavian  Journal  of 
Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

VanWely,  P.  (1970).  Design  and  disease.  Appl.  Ergon,  (y o\. 

1,  pp.  262-269). 

Potential  WMD 

Lower/upper  back  disorders. 

Question  -  23: 

I  lean  forward  continually  when  I  work  (e.g.,  when  sitting,  when 
standing,  when  pushing  carts,  etc.). 

Original  Question: 

Is  repeated  or  sustained  work  performed  when  the  back  is  flexed 
forward? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor  (e.g.,  “lean  forward”  replaces  “when 
the  back  is  flexed”),  question  simplified  to  focus  on  working  in 
an  awkward  static  posture. 

Risk  Factor: 

Reference 

Chaffin,  D.B.,  Andersson,  G.B.J.  (1984).  Occupational 
Biomechanics  (pp.  304).  John  Wiley  &  Sons,  New  York. 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scandinavian  Journal  of 
Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

Potential  WMD 

Lower/upper  back  disorders. 
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Question  -  24: 

The  personal  protective  equipment  or  clothing  that  I  wear  limits 
or  restricts  my  movement. 

Original  Question: 

N/A 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  to  reflect  the  additional  demands  that  may  be 
placed  on  the  body  (e.g.,  force)  due  to  PPE. 

Risk  Factor: 

Force  application,  awkward/non-neutral  body  segment  positions. 

Reference 

Akbarkhanzadeh,  F.,  Bisesi,  M.S.,  Rivas,  R.D.  (1995).  Comfort 
of  personal  protective  equipment.  Appl.  Ergon.  (Vol.  26,  No. 
3,pp.  195-198). 

Dunbar,  E.  (1993).  The  role  of  psychological  stress  and  prior 
experience  in  the  use  of  personal  protective  equipment.  J. 
Safety  Res.  (Vol.  24,  No.  3,pp.  181-187). 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 
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Question  -  25: 

I  perform  a  series  of  repetitive  tasks  or  back  movements  during 
the  course  of  my  work  (e.g.,  bending  forward,  backward,  or  to 
the  side,  or  twisting). 

Original  Question: 

Is  repeated  or  sustained  work  performed  when  the  back  is  flexed 
forward? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Simplified  the  question  to  place  focus  on  repeated  stressful 
movements  of  the  back,  eliminated  reference  to  sustained  work 
(now  in  Question  23),  described  the  types  of  back  movements  to 
be  considered. 

Risk  Factor: 

Asymmetrical  lifting,  twisting,  non-neutral  back  positions. 

Reference 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scandinavian  Journal  of 
Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

Fard,  H.,  and  Mital,  A.  (1993).  A  psychophysical  study  of  high 
and  very  high  frequency  manual  materials  handling  -  Part  I: 
Lifting  and  Lowering.  Int.  J.  Ind.  Ergon.  (Vol.  12,  pp.  127- 
141). 

Fard,  H.,  and  Mital,  A.  (1993).  A  psychophysical  study  of  high 
and  very  high  frequency  manual  materials  handling  -  Part  II: 
Carrying  and  Turning.  Int.  J.  Ind.  Ergon.  (Vol.  12,  pp.  143- 
156). 

Kumar,  S.  (1995)  Development  of  predictive  equations  for 
lifting  strength.  Appl.  Ergon.  (Vol.  26,  No.  5,  pp.  327-341). 

Mital,  A.,  Foononifard,  H.,  and  Brown,  M.L.  (1994,  Jime). 
Physical  fatigue  in  high  and  very  high  frequency  manual 
handling  -  perceived  exertion  and  physiological  indicators. 
Human  Factors  (Vol.  36,  No.  2,  pp.  219-23 1). 

Thomas,  R.G.,  van  Baar,  C.E.,  and  van  der  Stee,  M.J.  (1995). 
Baggage  handling:  Posture  and  the  design  of  conveyors. 

Appl.  Ergon.  (Vol.  26,  No.  2,  pp.  123-127). 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 
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Question  -  26: 

When  I  lift,  my  body  is  twisted  and/or  I  lift  quickly. 

Original  Question: 

Does  the  task  require  you  to  twist  or  bend  while  lifting/lowering 
or  pushing/pulling? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Simplified  question  to  eliminate  reference  to  bending  which  is 
included  in  other  questions,  combined  twisting  with  speed  of  lift 
(e.g.,  acceleration). 

Risk  Factor: 

Asymmetrical  lifting,  twisting,  high  speed  motions. 

Reference 

Fard,  H.,  and  Mital,  A.  (1993).  A  psychophysical  study  of  high 
and  very  high  frequency  manual  materials  handling  -  Part  I: 
Lifting  and  Lowering.  Int.  J.  Ind.  Ergon.  (Vol.  12,  pp.  127- 
141). 

Fard,  H.,  and  Mital,  A.  (1993).  A  psychophysical  study  of  high 
and  very  high  frequency  manual  materials  handling  -  Part  II: 
Carrying  and  Turning.  Int.  J.  Ind.  Ergon.  (Vol.  12,  pp.  143- 
156). 

Kumar,  S.  (1995).  Development  of  predictive  equations  for 
lifting  strength.  Appl.  Ergon.  (Vol.  26,  No.  5,  pp.  327-341). 

Mital,  A.,  Foononifard,  H.,  and  Brown,  M.L.  (1994,  June). 
Physical  fatigue  in  high  and  very  high  frequency  manual 
handling  -  perceived  exertion  and  physiological  indicators. 
Human  Factors  (Vol.  36,  No.  2,  pp.  219-231). 

Thomas,  R.G.,  van  Baar,  C.E.,  and  van  der  Stee,  M.J.  (1995). 
Baggage  handling:  Posture  and  the  design  of  conveyors. 

Appl  Ergon.  (Vol.  26,  No.  2,  pp.  123-127). 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 
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Question  -  27: 

I  can  feel  vibration  through  the  surface  that  I  stand  on  or  through 
my  seat. 

Original  Question: 

Do  you  work  on  jolting  surfaces  e.g.,  vibrating  floor,  ship  floor, 
vehicle  seat? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  reworded  to  focus  on  what  the 
employee  experiences  rather  than  an  aspect  of  the  workplace. 

Risk  Factor: 

Vibration,  unsupported  seating. 

Reference 

Wiktorin,  C.,  Karlqvist,  L.,  et  al.  (1993).  Validity  of  self- 
reported  exposures  to  work  postures  and  manual  materials 
handling.  Scand.  J.  Work  Environ.  Health  (Vol.  19,  pp.  208- 
214). 

Chaffin,  D.B.,  Andersson,  G.B.J.  (1984).  Occupational 
Biomechanics  (pp.  304).  New  York:  John  Wiley  &  Sons. 

Mattila,  M.,  Karwowski,  W.,  and  Vilkko,  M.  (1993,  December). 
Analysis  of  working  postures  in  hammering  tasks  on  building 
construction  sites  using  the  computerized  OWAS  method. 

(Vol.  24,  No.  6).  University  of  Louisville  and  Tampere 
University  of  Technology,  Finland. 

Potential  WMD 

Lower  back  disorders. 

Question  -  28: 

I  lift  and/or  carry  items  with  one  hand. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  asymmetric  loading  of  the  spine 
which  occurs  in  a  one  handed  lift/cany. 

Risk  Factor: 

Manual  materials  handling  of  heavy  loads. 

Reference 

Mital,  A.,  and  Asfour,  S.S.  (1983).  Maximum  frequencies 
acceptable  to  males  for  one-handed  lifting  in  the  sagital 
plane.  Human  Factors  (Vol.  25,  No.  5,  pp.  563-571). 

Mital,  A.,  Nicholson,  A.S.,  and  Ayoub,  M.M.  (1993).  A  Guide 
to  Manual  Materials  Handling.  London:  Taylor  &  Francis. 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 
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Job  Factor  Questions,  Research  Basis  for  Questions 
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Question  -  29: 

I  lift  or  handle  bulky  items. 

Original  Question: 

N/A  -  based  on  core  risk  factor. 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

New  question  added  to  reflect  Air  Force  work  situations 
involving  large  parts  (with  or  without  use  of  a  hoist). 

Risk  Factor: 

Manual  materials  handling  of  heavy  loads. 

Reference 

Garg,  A.,  Owen,  B.,  (1994).  Prevention  of  back  injuries  in 
healthcare  workers.  Int.  J.  Ind.  Ergon.  (Vol.  14,  pp.  315- 
331). 

Mital,  A.,  Nicholson,  A.S.,  Ayoub,  M.M.  (1993).  A  Guide  to 
Manual  Materials  Handling.  London:  Taylor  &  Francis. 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 

1  Question -30: 

I  lift  materials  that  weigh  more  than  25  pounds. 

Original  Question: 

N/A 

Validity  Testing 
Reported: 

N/A 

Rationale  for 
Change: 

Added  new  question  to  identify  lifting  situations  that  may  be 
present  in  administrative  areas  (when  sitting,  standing,  etc.). 

Risk  Factor: 

Manual  materials  handling  of  heavy  loads. 

Reference 

Mital,  A.,  and  Manivasagan,  I.  (1983).  Maximum  acceptable 
weight  of  lift  as  a  function  of  material  density,  center  of 
gravity  location,  hand  preference,  and  firequency.  Human 
Factors  (Vol.  25,  No.  1,  pp.  33-42). 

Mital,  A.,  Nicholson,  A.S.,  and  Ayoub,  M.M.  (1993).  A  Guide 
to  Manual  Materials  Handling.  London:  Taylor  &  Francis. 

Potential  WMD 

Lower/upper  back  disorders.  Tendonitis. 

Attachment  1-18 
133 


Job  Factor  Questions,  Research  Basis  for  Questions 
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1  Question -31: 

My  work  requires  that  I  kneel  or  squat. 

Original  Question: 

Does  your  work  involve  that  you  kneel  or  squat? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  first  person  verbiage. 

Risk  Factor: 

Prolonged  force  application. 

Reference 

Wiktorian,  et  al.  (1993).  Validity  of  self-reported  exposures  to 
work  postures  and  manual  materials  handling.  Scand  J. 

Work  Environ  Health  (Vol.  19,  pp.  208-214). 

Mattila,  M.,  Karwowski,  W.,  and  Vilkko,  M.  (1993,  December). 
Analysis  of  working  postures  in  hammering  tasks  on  building 
construction  sites  using  the  computerized  OWAS  method 
(Vol.  24,  No.  6).  University  of  Souisville  and  Tampere 
University  of  Technology,  Finland. 

OSEA  Draft  Ergonomic  Protection  Standard  (included  in  list  of 
signal  risk  factors). 

Potential  WMD 

Bursitis  of  the  knee. 

Question  -  32: 

I  must  constantly  move  or  apply  pressure  with  one  or  both  feet 
(e.g.,  using  foot  pedals,  driving,  etc.). 

Original  Question: 

Is  fatiguing  foot  pedal  work  performed? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

The  question  was  revised  to  provide  a  description  of  fatiguing 
work  to  eliminate  the  need  for  the  employee  to  make  a  fatigue 
judgment,  clarification  of  risk  factor. 

Risk  Factor: 

Static  fatigue-lower  limbs. 

Reference 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scand.  J.  of 

Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

Potential  WMD 

Lower  back  disorders.  Varicose  veins. 
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Question  -  33: 

When  I’m  sitting,  I  cannot  rest  both  feet  flat  on  the  floor. 

Original  Question: 

Are  the  foot/legs  unsupported  or  your  thighs  sloping  down  in  the 
front  (there  is  no  footrest  or  it  is  not  able  to  be  used)? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Clarification  of  risk  factor  (when  I’m  sitting),  simplification  of 
question. 

Risk  Factor: 

Unsupported  lumbar  region,  external  trauma  to  back  of  legs. 

Reference 

van  Wely  P.  (1970).  Design  and  disease.  Appl.  Ergon.  (Vol.  1, 
pp.  262-269). 

Reynolds,  J.L.,  Drury,  D.G.,  and  Broderick,  R.L.  (1994).  A  field 
methodology  for  the  control  of  musculoskeletal  injuries. 
Applied  Ergonomics  (Vol.  25,  No.  1,  pp.  3-16). 

Potential  WMD 

Lower  back  disorders. 

Question  -  34: 

I  stand  on  hard  surfaces. 

Original  Question: 

Is  the  standing  surface  hard  and  unsupported  (no  mat)? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Clarification  of  risk  factor,  removed  reference  to  “no  mat”  since 
floor  does  not  necessarily  have  to  be  equipped  with  a  mat  to  be 
an  acceptable  surface. 

Risk  Factor: 

Static  fatigue. 

Reference 

Konz,  S.  (1994).  Ergonomics  (Volume  37,  Number  4,  pp.  677). 

Reynolds,  J.L.,  Drury,  D.G.,  and  Broderick,  R.L.  (1994).  A  field 
methodology  for  the  control  of  musculoskeletal  injuries. 
Applied  Ergonomics  (Vol.  25,  No.  1,  pp.  3-16). 

Ryan,  G.A.  (1989).  Musculoskeletal  symptoms  in  supermarket 
workers.  Ergonomics  (Vol.  32,  No.  4,  pp.  359-371). 

Potential  WMD 

Varicose  veins.  Plantar  Fascitis,  Lower  back  disorders. 
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and  References,  and  Rationale  for  Question  Modification  (Contd.) 


1  Question  -  35: 

I  can  see  glare  on  my  computer  screen  or  work  surface. 

Original  Question: 

Is  there  glare  from  surface  reflections  or  other  light  sources 
which  affects  your  ability  to  see  your  work? 

Validity  Testing 
Reported: 

No. 

Rationale  for 
Change: 

Clarification  of  risk  factor,  simplification  of  question. 

Risk  Factor: 

Excessive  glare. 

Reference 

Reynolds,  J.L.,  Drury,  D.G.,  and  Broderick,  R.L.  (1994).  A  field 
methodology  for  the  control  of  musculoskeletal  injuries. 
Applied  Ergonomics  (Vol.  25,  No.  1,  pp.  3-16). 

American  National  Standards  Institute  (ANSI)/Human  Factors 
Society  Standard  100  (1988).  Human  Factors  Engineering  of 
Visual  Display  Terminal  Workstations  (pp.  11). 

Canadian  Standards  Association  (1989).  Office  Ergonomics:  A 
National  Standard  of  Canada  (pp.  56). 

Potential  WMD 

Eye  fatigue. 

Question  -  36: 

It  is  difficult  to  hear  a  person  on  the  phone  or  to  concentrate 
because  of  other  activity,  voices,  or  noise  in/near  my  work  area. 

Original  Question: 

Are  there  noises  or  soimds  that  distract  you  from  your  job? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Provided  examples  of  “being  distracted”  to  provide  context  for 
the  question,  placed  focus  on  being  distracted  rather  than  on 
noise. 

Risk  Factor: 

Lapses  in  concentration. 

Reference 

Kjellberg,  A.,  and  Landstrom,  U.  (1994).  Noise  in  the  office: 

Part  I  -  Guidelines  for  the  practitioner.  Int.  J.  Ind.  Ergon. 

(Vol.  14,  pp.  87-91). 

Steelcase  “Healthy  Office.” 

Kjellberg,  A.,  and  Landstrom,  U.  (1994).  Noise  in  the  office: 

Part  II  -  The  scientific  basis  (knowledge  base)  for  the  guide. 

Int.  J.  Ind.  Ergon.  (Vol.  14,  pp.  93-118). 

Potential  WMD 

N/A 
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Question  -  37: 

I  must  look  at  the  monitor  screen  constantly  so  that  I  do  not  miss 
important  information  (radar  scope). 

Original  Question: 

Are  there  high  demands  on  visual  capacity? 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Clarification  of  risk  factor,  eliminated  the  need  for  the  employee 
to  decide  what  “high”  is,  provided  an  example  of  a  job 
characteristic  (e.g.,  look  at  the  screen  constantly)  which  suggests 
high  demands  on  visual  capacity. 

Risk  Factor: 

Vigilance  tasks. 

Reference 

Bergqvist,  U.  (1995).  Video  Display  Terminal  work  -  A 
perspective  on  long  term  changes.  Int.  J.  Ind.  Ergon.  (Vol. 

16,  pp.  201-209). 

Kemmlert,  K.  (1994).  A  Method  Assigned  for  the  Identification 
of  Ergonomic  Hazards  -  PLIBEL.  Scand.  J.  of 

Rehabilitative  Medicine  (Vol.  26,  pp.  1-21). 

Potential  WMD 

Eye  fatigue,  head/neck  disorders. 

Question  -  38: 

It  is  difficult  to  see  what  I  am  working  with  (monitor,  paper, 
parts,  etc.). 

Original  Question: 

Is  the  total  lighting  level  inadequate  at  your  work  area? 

Validity  Testing 
Reported: 

No. 

Rationale  for 
Change: 

Clarification  of  risk  factor,  eliminated  the  need  for  the  employee 
to  decide  what  is  adequate. 

Risk  Factor: 

Poor  illumination. 

Reference 

Reynolds,  J.L.,  Drury,  D.G.,  and  Broderick,  R.L.  (1994).  A  field 
methodology  for  the  control  of  musculoskeletal  injuries. 
Applied  Ergonomics  (Vol.  25,  No.  1,  pp.  3-16). 

American  National  Standards  Institute  (ANSI)/Human  Factors 
Society  Standard  100  (1988).  Human  Factors  Engineering  of 
Visual  Display  Terminal  Workstations  (pp.  11). 

Canadian  Standards  Association  (1989).  Office  Ergonomics:  A 
National  Standard  of  Canada  (pp.  56). 

Potential  WMD 

Eye  fatigue,  head/neck  disorders. 
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Questions  39: 

I  often  feel  unclear  on  what  the  scope  and  responsibilities  of  my 
job  are. 

Original  Question: 

Being  unclear  on  just  what  the  scope  and  responsibilities  of  your 
job  are. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  rest  of  Survey  -  changed  sentence  structure  to  first 
person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  & 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potential  WMD 

N/A 

Questions  40: 

I  often  feel  that  I  have  too  heavy  of  workload,  one  that  I  could 
not  possibly  finish  during  an  ordinary  workday. 

Original  Question: 

Feeling  that  you  have  too  heavy  a  work  load,  one  that  you  can’t 
possibly  finish  during  an  ordinary  workday. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  rest  of  Survey  -  changed  sentence  structure  to  first 
person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  & 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potential  WMD 

N/A 
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Questions  41: 

I  often  feel  that  I  will  not  be  able  to  satisfy  the  conflicting 
demands  of  various  people  aroimd  me. 

Original  Question: 

Thinking  that  you’ll  not  be  able  to  satisfy  the  conflicting 
demands  of  various  people  over  you. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  rest  of  Siuvey  -  changed  sentence  structure  to  first 
person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  &, 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potnetial  WMD 

N/A 

Questions  42: 

I  often  find  myself  unable  to  get  information  needed  to  carry  out 
my  job. 

Original  Question: 

The  fact  that  you  can’t  get  information  needed  to  carry  out  your 
job. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  rest  of  Survey  -  changed  sentence  structure  to  first 
person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  & 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potential  WMD 

N/A 
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Questions  43: 

I  often  do  not  know  what  my  supervisor  thinks  of  me,  how 
he/she  evaluates  my  performance. 

Original  Question: 

Not  knowing  what  your  superviosr  thinnks  of  you,  how  he 
evaluates  your  performance. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  rest  of  Survey  -  changed  sentence  structure  to  first 
person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  & 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potential  WMD 

N/A 

Questions  44: 

I  often  think  that  the  amount  of  work  I  have  to  do  may  interfere 
with  how  well  it’s  done. 

Original  Question: 

Thinking  that  the  amount  of  work  you  have  to  do  may  interfere 
vsdth  how  well  it  gets  done. 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Better  fit  with  the  rest  of  the  Survey  -  changed  sentence  structure 
to  first  person. 

Risk  Factor: 

Psychosocial  Stressor. 

Reference 

Kahn,  R.  L.,  Wolfe,  D.  M.,  Quinn,  R.  P.,  Snoek,  J.  D.  & 

Rosenthal,  R.  A.  (1964).  Organizational  stress:  studies  in  role 
conflict  and  ambiguity.  New  York.  John  Wiley  and  Sons, 

Inc. 

Potential  WMD 

N/A 
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Questions  45: 

How  would  you  describe  the  physical  effort  required  of  you  job? 

Original  Question: 

Same 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

N/A 

Risk  Factor: 

Whole  body  exertion,  fatigue. 

Reference 

Borg,  G.  (1970).  Perceived  exertion  as  an  indicator  of  somatic 
stress.  Scandanavian  Journal  of  Rehab.  Medicine  (vol  2,  pp. 
92-98). 

Potential  WMD 

N/A 

Questions  46, 49, 

52, 55, 58: 

In  the  past  12  months  have  you  experienced  any  discomfort 
fatigue,  numbness  or  pain  that  relates  to  your  job? 

Original  Question: 

Have  you  at  any  time  during  the  last  12  months  had  trouble 
(such  as  ache,  pain,  discomfort,  numbness)  in: 

Validity  Testing 
Reported: 

Yes 

Rationale  for 
Change: 

Increase  focus  on  work  related  discomfort. 

Risk  Factor: 

N/A 

Reference 

Dickinson,  C.  E.,  Campion,  K.,  Foster,  A.  F.,  Newman,  S.  J., 
O’Rourke,  A.  M.  T.,  &  Thomas,  P.  G.  (1992).  Questioimaire 
development:  an  examination  of  the  Nordic  Musculoskeletal 
Questioimaire.  Applied  Ergonomics  (vol  23,  No  3,  pp  197- 
201). 

Potential  WMD 

N/A 

Questions  47, 50, 

53, 56, 59: 

How  often  do  you  experience  discomfort,  numbness  or  pain  in 
this  region  of  the  body? 

Original  Question: 

During  the  last  year ,  how  may  different  times  have  you  had  this 
problem? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Improve  clarity,  question  was  skipped  if  no  discomfort  was 
experienced  during  the  last  year. 

Risk  Factor: 

N/A 

Reference 

ANSI  Z-365  (1993,  June  4).  Sample  surveillance  tools. 

Potential  WMD 

N/A 
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Questions  48, 51, 

54, 57, 60: 

On  average,  how  severe  is  the  discomfort,  fatigue,  numbness,  or 
pain  in  this  region  of  the  body? 

Original  Question: 

According  to  the  scale  of  0-5  at  the  right,  how  would  you  rate 
this  problem  right  now? 

Validity  Testing 
Reported: 

No 

Rationale  for 
Change: 

Provided  descriptor  in  answer  to  enhace  clarity,  provide  focus  on 
routine  exposure  (average)  as  compared  to  momentary  exposure. 

Risk  Factor: 

N/A 

Reference 

ANSI  Z-365  (1993,  Jime  4).  Sample  surveillance  tools. 

Potential  WMD 

N/A 
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Agreements  (N=40) 


Yes 

No 

Shoulder/Neck 

5 

27 

HandAVrist 

8 

27 

Back/Torso 

1 

35 

Legs/Feet 

13 

19 

Head/Eyes 

6 

26 

Shoulder 

8 

25 

Wrist 

8 

27 

Back 

3 

30 

Legs 

1 

37 

Hand 

6 

28 
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Agreements  (N=40) 

0-2 

Hours 

Question  1 

24 

Question  2 

40 

Question  3 

35 

Question  4 

35 

Question  5 

13 

Question  6 

12 

Question  7 

33 

Question  8 

17 

Question  9 

19 

Question  10 

30 

Question  11 

9 

Question  12 

30 

Question  13 

37 

Question  14 

34 

Question  15 

27 

Question  16 

26 

Question  17 

39 

Question  18 

30 

Question  19 

37 

Question  20 

38 

Question  21 

25 

Question  22 

40 

Question  23 

18 

Question  24 

36 

Question  25 

29 

Question  26 

39 

Question  27 

38 

Question  28 

39 

Question  29 

37 

Question  30 

36 

Question  31 

35 

Question  32 

32 

Question  33 

31 

Question  34 

23 

Question  35 

23 

Question  36 

25 

Question  37 

37 

Question  38 

35 
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Question  47 


Question  50 
Question  53 


Question  56 


Question  59 


Question  48 


Question  51 


Question  54 


Question  57 


Question  60 


Agreements  (N=40) 


Moderate 


3 


Severe 


1 
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Agreements:  Q45  18/40 


6 

7 

8 

9 

11 

12 

13 

14 

15 

1 

3 

1 

4 

4 

1 

4 

1 

1 
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